Jul 10 15:52:59.776468 ip-10-0-135-156 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 10 15:52:59.776481 ip-10-0-135-156 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 10 15:52:59.776492 ip-10-0-135-156 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 10 15:52:59.776792 ip-10-0-135-156 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 10 15:53:09.981673 ip-10-0-135-156 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 10 15:53:09.981695 ip-10-0-135-156 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot db820461c5054f77ad54ec0bca9d267f --
Jul 10 15:55:11.570019 ip-10-0-135-156 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 15:55:11.970044 ip-10-0-135-156 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 15:55:11.970044 ip-10-0-135-156 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 15:55:11.970044 ip-10-0-135-156 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 15:55:11.970044 ip-10-0-135-156 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 15:55:11.970044 ip-10-0-135-156 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 15:55:11.970754 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.970056    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 15:55:11.976728 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976703    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 15:55:11.976728 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976726    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 15:55:11.976728 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976731    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976734    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976737    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976741    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976743    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976746    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976749    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976752    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976755    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976758    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976761    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976764    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976769    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976772    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976775    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976778    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976780    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976783    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976785    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 15:55:11.976832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976788    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976790    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976792    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976795    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976798    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976801    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976803    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976806    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976809    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976811    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976814    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976816    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976819    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976822    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976825    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976828    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976832    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976836    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 15:55:11.977318 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976839    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976844    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976848    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976851    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976854    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976857    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976860    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976863    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976865    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976868    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976870    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976873    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976876    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976878    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976881    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976885    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976889    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976892    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976895    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976898    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 15:55:11.977777 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976901    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976903    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976906    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976909    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976912    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976914    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976916    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976919    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976921    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976925    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976928    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976930    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976934    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976936    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976940    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976942    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976945    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976948    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976950    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976953    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 15:55:11.978290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976955    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976958    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976961    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976963    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976965    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976968    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976971    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976973    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976976    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976978    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976981    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976983    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976986    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976988    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976991    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976993    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976996    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.976998    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977001    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977003    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 15:55:11.978792 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977806    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977815    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977819    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977824    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977828    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977831    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977835    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977838    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977841    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977844    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977846    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977849    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977852    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977854    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977857    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977859    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977861    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977864    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977866    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977869    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 15:55:11.979297 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977871    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977874    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977876    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977880    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977883    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977887    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977890    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977892    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977896    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977900    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977903    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977907    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977909    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977912    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977914    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977918    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977920    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977923    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977926    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 15:55:11.979812 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977929    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977931    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977934    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977936    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977939    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977942    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977944    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977947    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977949    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977952    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977954    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977957    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977959    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977961    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977964    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977967    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977969    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977972    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977974    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 15:55:11.980310 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977977    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977979    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977983    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977985    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977988    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977990    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977992    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977995    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.977998    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978000    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978002    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978005    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978007    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978010    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978012    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978015    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978017    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978019    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978022    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978024    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 15:55:11.980787 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978027    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978030    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978032    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978035    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978037    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978039    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978042    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978044    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978047    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978049    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978051    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978054    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978056    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978059    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978061    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978064    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978066    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978068    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978071    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978074    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 15:55:11.981314 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978076    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978469    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978474    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978477    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978479    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978482    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978485    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978488    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978491    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978493    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978496    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978498    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978501    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978504    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978506    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978512    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978514    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978517    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978520    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978522    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 15:55:11.981831 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978525    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978529    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978533    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978536    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978539    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978542    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978544    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978547    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978550    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978553    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978556    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978559    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978561    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978564    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978566    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978569    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978571    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978574    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978576    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 15:55:11.982320 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978579    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978581    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978584    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978587    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978589    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978591    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978594    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978596    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978600    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978602    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978605    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978607    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978610    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978613    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978615    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978618    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978620    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978623    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978625    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 15:55:11.982803 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978627    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978630    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978632    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978634    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978637    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978639    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978642    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978644    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978647    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978649    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978651    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978654    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978656    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978658    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978662    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978664    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978667    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978669    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978671    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 15:55:11.983269 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978673    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978695    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978700    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978703    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978706    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978710    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978713    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978715    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978718    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978720    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978722    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978725    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978727    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978730    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978733    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978735    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978738    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978740    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978743    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 15:55:11.983754 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978745    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978748    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978750    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.978753    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978841    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978854    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978863    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978874    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978879    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978883    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978887    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978891    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978894    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978897    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978901    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978905    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978908    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978911    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978916    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978919    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978921    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978924    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978929    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978932    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 15:55:11.984218 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978935    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978938    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978941    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978945    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978948    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978951    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978954    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978957    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978960    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978963    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978966    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978969    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978973    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978976    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978978    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978981    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978984    2567 flags.go:64] FLAG: --enable-server="true"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978987    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978991    2567 flags.go:64] FLAG: --event-burst="100"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978994    2567 flags.go:64] FLAG: --event-qps="50"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.978997    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979000    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979003    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979007    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 15:55:11.984819 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979010    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979013    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979018    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979020    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979024    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979027    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979029    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979032    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979035    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979037    2567 flags.go:64] FLAG: --feature-gates=""
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979041    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979044    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979047    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979050    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979053    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979056    2567 flags.go:64] FLAG: --help="false"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979059    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-135-156.ec2.internal"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979062    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979064    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979067    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979071    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979074    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979077    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 15:55:11.985443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979079    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979082    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979085    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979088    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979092    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979094    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979097    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979100    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979103    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979106    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979108    2567 flags.go:64] FLAG: --lock-file=""
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979112    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979115    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979118    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979123    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979126    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979129    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979131    2567 flags.go:64] FLAG: --logging-format="text"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979134    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979137    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979140    2567 flags.go:64] FLAG: --manifest-url=""
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979143    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979148    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979150    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979154    2567 flags.go:64] FLAG: --max-pods="110"
Jul 10 15:55:11.986136 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979157    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979160    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979163    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979166    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979169    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979172    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979175    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979182    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979185    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979189    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979191    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979195    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979200    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979203    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979206    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979210    2567 flags.go:64] FLAG: --port="10250"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979212    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979215    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f5656bce1ac567d1"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979218    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979222    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979225    2567 flags.go:64] FLAG: --register-node="true"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979228    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979231    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979234    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 10 15:55:11.986796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979237    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979240    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979244    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979247    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979250    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979253    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979255    2567 flags.go:64] FLAG: --runonce="false"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979258    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979261    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979264    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979266    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979269    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979273    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979276    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979279    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979281    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979284    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979287    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979289    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979292    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979295    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979298    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979304    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979308    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979311    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 15:55:11.987398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979315    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979317    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979322    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979325    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979328    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979331    2567 flags.go:64] FLAG: --v="2"
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979335    2567 flags.go:64] FLAG: --version="false"
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979339    2567 flags.go:64] FLAG: --vmodule=""
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979342    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.979345    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979485    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979489    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979493    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979495    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979498    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979501    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979503    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979506    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979509    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979511    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979513    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 15:55:11.988031 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979516    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979518    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979520    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979523    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979525    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979528    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979531    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979538    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979540    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979544    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979547    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979549    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979552    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979556    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979558    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979560    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979563    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979565    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979568    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979570    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 15:55:11.988543 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979573    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979577    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979580    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979583    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979585    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979588    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979591    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979594    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979596    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979599    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979602    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979604    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979606    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979609    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979611    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979614    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979616    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979619    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979621    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979624    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 15:55:11.989058 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979627    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979630    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979633    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979635    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979637    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979641    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979644    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979646    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979648    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979651    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979655    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979658    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979661    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979664    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979666    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979669    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979671    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979674    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979693    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 15:55:11.990077 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979696    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979698    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979701    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979703    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979706    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979708    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979710    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979713    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979715    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979718    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979720    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979723    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979725    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979728    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979732    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979735    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979737    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979740    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979744    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979746    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 15:55:11.990647 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979750    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979754    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979757    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979760    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979763    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979766    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979768    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979771    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.979773    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.980292    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.987494    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.987514    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.988486    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.988494    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988568    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 15:55:11.991290 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988573    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988576    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988579    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988582    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988585    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988588    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988591    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988593    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988596    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988598    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988602    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988605    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988608    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988613    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988617    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988620    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988622    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988625    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988628    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988630    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 15:55:11.991703 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988633    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988636    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988638    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988641    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988644    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988646    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988649    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988651    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988654    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988657    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988660    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988662    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988665    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988667    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988670    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988672    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988675    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988693    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988696    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988699    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 15:55:11.992203 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988702    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988704    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988707    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988710    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988713    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988717    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988720    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988723    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988725    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988728    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988731    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988733    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988736    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988739    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988741    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988745    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988747    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988750    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988753    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988756    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 15:55:11.992712 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988758    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988761    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988764    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988767    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988770    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988772    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988775    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988778    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988780    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988783    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988785    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988787    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988790    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988792    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988795    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988798    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988800    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988804    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988806    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 15:55:11.993207 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988809    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988813    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988817    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988820    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988823    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988826    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988828    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988831    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988834    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988837    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988840    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988844    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988847    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988851    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988854    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988857    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988860    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988862    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 15:55:11.993704 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.988865    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.988870    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989494    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989519    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989525    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989530    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989535    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989548    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989553    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989558    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989563    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989569    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989573    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989578    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 15:55:11.994164 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989583    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989590    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989595    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989600    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989607    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989611    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989625    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989632    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989636    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989640    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989647    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989651    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989656    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989662    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989666    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989671    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989698    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989703    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989708    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989718    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 15:55:11.994512 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989722    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989729    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989734    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989738    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989743    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989747    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989751    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989756    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989760    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989764    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989768    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989777    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989782    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989786    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989791    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989795    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989800    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989805    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989809    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989814    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 15:55:11.995032 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989818    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989824    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989831    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989841    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989846    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989851    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989858    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989865    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989871    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989877    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989881    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989887    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989892    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989897    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989903    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989909    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989919    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989925    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989930    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 15:55:11.995510 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989935    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989940    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989944    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989948    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989953    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989958    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989963    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989967    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989971    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989982    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989986    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989990    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.989998    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990004    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990009    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990013    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990018    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990023    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990028    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 15:55:11.995986 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990032    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990038    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990042    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990052    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990057    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990062    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990067    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990071    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:11.990075    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.990085    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.990697    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.992730    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.994085    2567 server.go:1019] "Starting client certificate rotation"
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.994188    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 15:55:11.996451 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.994815    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 15:55:11.996900 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.996731    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 15:55:11.998721 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:11.998706    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 15:55:12.006863 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.006841    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 10 15:55:12.009031 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.009015    2567 log.go:25] "Validated CRI v1 image API"
Jul 10 15:55:12.010369 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.010353    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 15:55:12.013568 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.013536    2567 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8cefa52c-0fd5-4623-b65e-f9579d1de387:/dev/nvme0n1p3 c8780242-c24d-449a-85c4-5fe7234b1f30:/dev/nvme0n1p4]
Jul 10 15:55:12.013658 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.013567    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 15:55:12.019045 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.018908    2567 manager.go:216] Machine: {Timestamp:2026-07-10 15:55:12.017524309 +0000 UTC m=+0.327780907 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100045 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f57138fd7918a42527af964ff46ba SystemUUID:ec2f5713-8fd7-918a-4252-7af964ff46ba BootID:db820461-c505-4f77-ad54-ec0bca9d267f Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:83:bc:72:b1:df Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:83:bc:72:b1:df Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:40:4d:2a:e2:54 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 15:55:12.019045 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.019036    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 15:55:12.019191 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.019160    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 15:55:12.022275 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.022250    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 15:55:12.022426 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.022282    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-156.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 15:55:12.023034 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.023024    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 15:55:12.023072 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.023038    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 15:55:12.023072 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.023045    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 15:55:12.023712 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.023701    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 15:55:12.024876 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.024866    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 15:55:12.027617 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.027602    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 15:55:12.027762 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.027752    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 15:55:12.027795 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.027785    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 15:55:12.027830 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.027799    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 15:55:12.027830 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.027810    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 15:55:12.027830 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.027819    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 10 15:55:12.027830 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.027827    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 15:55:12.028269 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.028243    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 15:55:12.028269 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.028268    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 15:55:12.030637 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.030602    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 15:55:12.032171 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.032153    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 15:55:12.034043 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034019    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 15:55:12.034127 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034071    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 15:55:12.034293 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034280    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 15:55:12.034334 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034302    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 15:55:12.034334 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034313    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 15:55:12.034334 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034320    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 15:55:12.034334 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034329    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 15:55:12.034334 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034334    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 15:55:12.034939 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034930    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 15:55:12.034939 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034939    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 15:55:12.035008 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034947    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 15:55:12.035008 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034953    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 15:55:12.035008 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034962    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 15:55:12.035008 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.034969    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 15:55:12.036870 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.036859    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 15:55:12.036870 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.036871    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 15:55:12.040289 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.040273    2567 server.go:1268] "Started kubelet"
Jul 10 15:55:12.040399 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.040350    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 15:55:12.040477 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.040386    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 15:55:12.040566 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.040503    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 15:55:12.040846 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.040826    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-156.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 15:55:12.041327 ip-10-0-135-156 systemd[1]: Started Kubernetes Kubelet.
Jul 10 15:55:12.041443 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.041357    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 15:55:12.041443 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.041362    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-156.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 15:55:12.041576 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.041553    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 15:55:12.042808 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.042793    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 15:55:12.044703 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.044669    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 15:55:12.046656 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.046636    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 15:55:12.046656 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.046646    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 15:55:12.046818 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.046753    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:12.047200 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.047181    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 15:55:12.047200 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.047184    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 15:55:12.047354 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.047210    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 15:55:12.047354 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.047325    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 15:55:12.047354 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.047334    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 15:55:12.047880 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.047861    2567 factory.go:55] Registering systemd factory
Jul 10 15:55:12.047880 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.047883    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 10 15:55:12.048090 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.047070    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-156.ec2.internal.18c0f8b5c668d75a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-156.ec2.internal,UID:ip-10-0-135-156.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-156.ec2.internal,},FirstTimestamp:2026-07-10 15:55:12.040245082 +0000 UTC m=+0.350501679,LastTimestamp:2026-07-10 15:55:12.040245082 +0000 UTC m=+0.350501679,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-156.ec2.internal,}"
Jul 10 15:55:12.048174 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.048119    2567 factory.go:153] Registering CRI-O factory
Jul 10 15:55:12.048174 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.048134    2567 factory.go:223] Registration of the crio container factory successfully
Jul 10 15:55:12.048246 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.048206    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 15:55:12.048246 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.048229    2567 factory.go:103] Registering Raw factory
Jul 10 15:55:12.048246 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.048241    2567 manager.go:1196] Started watching for new ooms in manager
Jul 10 15:55:12.048731 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.048699    2567 manager.go:349] Starting recovery of all containers
Jul 10 15:55:12.057048 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.056997    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b9hx6"
Jul 10 15:55:12.057552 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.057523    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 15:55:12.057859 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.057650    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-156.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 15:55:12.060326 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.060307    2567 manager.go:354] Recovery completed
Jul 10 15:55:12.065435 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.065421    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 15:55:12.067338 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.067320    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 15:55:12.067421 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.067357    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 15:55:12.067421 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.067374    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientPID"
Jul 10 15:55:12.067989 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.067968    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b9hx6"
Jul 10 15:55:12.068060 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.068007    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 15:55:12.068060 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.068017    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 15:55:12.068060 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.068039    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 15:55:12.068060 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.068043    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 15:55:12.068760 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.068745    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 15:55:12.068820 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.068775    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 15:55:12.068820 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.068784    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientPID"
Jul 10 15:55:12.069241 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.069230    2567 policy_none.go:49] "None policy: Start"
Jul 10 15:55:12.069283 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.069246    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 15:55:12.069283 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.069258    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 15:55:12.069910 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.069828    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-156.ec2.internal.18c0f8b5c8064607  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-156.ec2.internal,UID:ip-10-0-135-156.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-156.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-156.ec2.internal,},FirstTimestamp:2026-07-10 15:55:12.067339783 +0000 UTC m=+0.377596384,LastTimestamp:2026-07-10 15:55:12.067339783 +0000 UTC m=+0.377596384,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-156.ec2.internal,}"
Jul 10 15:55:12.070267 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.070253    2567 policy_none.go:47] "Start"
Jul 10 15:55:12.111375 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.111356    2567 manager.go:337] "Starting Device Plugin manager"
Jul 10 15:55:12.111512 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.111393    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 15:55:12.111512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.111405    2567 server.go:87] "Starting device plugin registration server"
Jul 10 15:55:12.111724 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.111711    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 15:55:12.111781 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.111724    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 15:55:12.111874 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.111854    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 15:55:12.111954 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.111942    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 15:55:12.111996 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.111956    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 15:55:12.112523 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.112507    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 15:55:12.112616 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.112552    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:12.212158 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.212125    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 15:55:12.214320 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.214299    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 15:55:12.214443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.214333    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 15:55:12.214443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.214349    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientPID"
Jul 10 15:55:12.214443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.214381    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.222751 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.222697    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.222751 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.222725    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-156.ec2.internal\": node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:12.230705 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.230654    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 15:55:12.232013 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.231994    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 15:55:12.232088 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.232041    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 15:55:12.232088 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.232075    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 15:55:12.232179 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.232117    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 15:55:12.234229 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.234212    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 15:55:12.242019 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.242000    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:12.332437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.332405    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-156.ec2.internal"]
Jul 10 15:55:12.332564 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.332486    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 15:55:12.333461 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.333445    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 15:55:12.333544 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.333478    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 15:55:12.333544 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.333492    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientPID"
Jul 10 15:55:12.335164 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.335150    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 15:55:12.335285 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.335269    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.335337 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.335307    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 15:55:12.335954 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.335938    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 15:55:12.335954 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.335953    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 15:55:12.336066 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.335968    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 15:55:12.336066 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.335973    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 15:55:12.336066 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.335978    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientPID"
Jul 10 15:55:12.336066 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.335983    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientPID"
Jul 10 15:55:12.337219 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.337205    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.337262 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.337249    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 15:55:12.337965 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.337951    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 15:55:12.338029 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.337979    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 15:55:12.338029 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.337996    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeHasSufficientPID"
Jul 10 15:55:12.342205 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.342189    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:12.348618 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.348600    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23961ef0df5a0f6fd14429a033775076-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal\" (UID: \"23961ef0df5a0f6fd14429a033775076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.348720 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.348626    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23961ef0df5a0f6fd14429a033775076-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal\" (UID: \"23961ef0df5a0f6fd14429a033775076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.348720 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.348645    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6a91c7d5b3d2028299be24baf3606bd6-config\") pod \"kube-apiserver-proxy-ip-10-0-135-156.ec2.internal\" (UID: \"6a91c7d5b3d2028299be24baf3606bd6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.362625 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.362595    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-156.ec2.internal\" not found" node="ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.367352 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.367336    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-156.ec2.internal\" not found" node="ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.442816 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.442765    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:12.449431 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.449404    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23961ef0df5a0f6fd14429a033775076-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal\" (UID: \"23961ef0df5a0f6fd14429a033775076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.449431 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.449435    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6a91c7d5b3d2028299be24baf3606bd6-config\") pod \"kube-apiserver-proxy-ip-10-0-135-156.ec2.internal\" (UID: \"6a91c7d5b3d2028299be24baf3606bd6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.449618 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.449451    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23961ef0df5a0f6fd14429a033775076-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal\" (UID: \"23961ef0df5a0f6fd14429a033775076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.449618 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.449481    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/23961ef0df5a0f6fd14429a033775076-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal\" (UID: \"23961ef0df5a0f6fd14429a033775076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.449618 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.449507    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23961ef0df5a0f6fd14429a033775076-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal\" (UID: \"23961ef0df5a0f6fd14429a033775076\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.449618 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.449517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6a91c7d5b3d2028299be24baf3606bd6-config\") pod \"kube-apiserver-proxy-ip-10-0-135-156.ec2.internal\" (UID: \"6a91c7d5b3d2028299be24baf3606bd6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.543301 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.543216    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:12.643764 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.643720    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:12.664247 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.664221    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.670235 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.670205    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:12.744697 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.744656    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:12.845240 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.845173    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:12.945735 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:12.945706    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:12.993420 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.993388    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 15:55:12.993964 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:12.993536    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 15:55:13.045998 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:13.045965    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:13.071435 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:13.071399    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 15:50:13 +0000 UTC" deadline="2028-01-22 19:25:11.052569298 +0000 UTC"
Jul 10 15:55:13.071435 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:13.071431    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13467h29m57.981142541s"
Jul 10 15:55:14.313484 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:14.313453    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:14.326882 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:14.326861    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 15:55:14.355694 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:14.355663    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23961ef0df5a0f6fd14429a033775076.slice/crio-43c1079325a8be72ff7e9e04aad32a5072b279a88e929dc99246a503b863b52f WatchSource:0}: Error finding container 43c1079325a8be72ff7e9e04aad32a5072b279a88e929dc99246a503b863b52f: Status 404 returned error can't find the container with id 43c1079325a8be72ff7e9e04aad32a5072b279a88e929dc99246a503b863b52f
Jul 10 15:55:14.360218 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:14.360194    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a91c7d5b3d2028299be24baf3606bd6.slice/crio-25950473c7da664092fb4bf429e661c741f17f41abcc13d61e562200de9877c1 WatchSource:0}: Error finding container 25950473c7da664092fb4bf429e661c741f17f41abcc13d61e562200de9877c1: Status 404 returned error can't find the container with id 25950473c7da664092fb4bf429e661c741f17f41abcc13d61e562200de9877c1
Jul 10 15:55:14.361096 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:14.361079    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 15:55:14.402573 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:14.402517    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-156.ec2.internal" event={"ID":"6a91c7d5b3d2028299be24baf3606bd6","Type":"ContainerStarted","Data":"25950473c7da664092fb4bf429e661c741f17f41abcc13d61e562200de9877c1"}
Jul 10 15:55:14.403530 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:14.403510    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal" event={"ID":"23961ef0df5a0f6fd14429a033775076","Type":"ContainerStarted","Data":"43c1079325a8be72ff7e9e04aad32a5072b279a88e929dc99246a503b863b52f"}
Jul 10 15:55:14.413698 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:14.413670    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:14.514223 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:14.514147    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:14.614511 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:14.614476    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:14.715316 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:14.715282    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-156.ec2.internal\" not found"
Jul 10 15:55:14.767376 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:14.767283    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 15:55:14.814522 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:14.814489    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:14.825242 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:14.825213    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 15:55:14.827111 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:14.827078    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal"
Jul 10 15:55:14.837789 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:14.837759    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 15:55:15.163845 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.163747    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 15:55:15.167169 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.167144    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 15:55:15.196487 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.196456    2567 apiserver.go:52] "Watching apiserver"
Jul 10 15:55:15.208437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.207961    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 15:55:15.209287 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.209250    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-156.ec2.internal","openshift-cluster-node-tuning-operator/tuned-k5759","openshift-dns/node-resolver-htc8d","openshift-image-registry/node-ca-kdsx4","openshift-multus/multus-additional-cni-plugins-lwpkp","openshift-network-operator/iptables-alerter-4czq2","openshift-ovn-kubernetes/ovnkube-node-qz2jw","kube-system/global-pull-secret-syncer-l5nkc","kube-system/konnectivity-agent-hw282","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal","openshift-multus/multus-frcsg","openshift-multus/network-metrics-daemon-lh5cf","openshift-network-diagnostics/network-check-target-xbfbt"]
Jul 10 15:55:15.211189 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.211163    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.212443 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.212422    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:15.212563 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.212500    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:15.214529 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.213309    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-htc8d"
Jul 10 15:55:15.214529 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.213846    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 15:55:15.214529 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.214185    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 15:55:15.214529 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.214188    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7bhng\""
Jul 10 15:55:15.214795 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.214559    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 15:55:15.214795 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.214605    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 15:55:15.214895 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.214828    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kdsx4"
Jul 10 15:55:15.214895 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.214858    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 15:55:15.214991 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.214966    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 15:55:15.215069 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.215047    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 15:55:15.216011 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.215985    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 15:55:15.217121 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.217101    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 15:55:15.217333 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.217311    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jz8n7\""
Jul 10 15:55:15.219152 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.217922    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 15:55:15.219152 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.218118    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 15:55:15.219152 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.218186    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.219152 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.218295    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fg4m9\""
Jul 10 15:55:15.219152 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.218186    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:15.219152 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.218508    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4czq2"
Jul 10 15:55:15.219152 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.218623    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 15:55:15.219890 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.219627    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:15.219890 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.219699    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:15.221318 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.221286    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 15:55:15.221776 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.221553    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 15:55:15.221855 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.221793    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 15:55:15.222001 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.221986    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 15:55:15.222183 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.222169    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 15:55:15.222604 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.222582    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7b92z\""
Jul 10 15:55:15.222720 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.222628    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 15:55:15.222720 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.222693    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 15:55:15.222841 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.222756    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f66gp\""
Jul 10 15:55:15.222841 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.222792    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 15:55:15.222968 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.222876    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 15:55:15.222968 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.222945    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kk29t\""
Jul 10 15:55:15.223064 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.222986    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.223256 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.223235    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 15:55:15.223711 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.223689    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.225197 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225173    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-log-socket\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225297 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225210    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-cni-bin\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225297 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225236    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrzxc\" (UniqueName: \"kubernetes.io/projected/112868fd-c401-4f97-81a9-6a1355d9046b-kube-api-access-mrzxc\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225428 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225301    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-kubelet\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225428 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225335    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-slash\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225428 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225360    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/112868fd-c401-4f97-81a9-6a1355d9046b-env-overrides\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225428 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225386    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fc05a30-9841-4627-a43e-5591964cce62-host\") pod \"node-ca-kdsx4\" (UID: \"9fc05a30-9841-4627-a43e-5591964cce62\") " pod="openshift-image-registry/node-ca-kdsx4"
Jul 10 15:55:15.225635 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225446    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:15.225635 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225469    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9fc05a30-9841-4627-a43e-5591964cce62-serviceca\") pod \"node-ca-kdsx4\" (UID: \"9fc05a30-9841-4627-a43e-5591964cce62\") " pod="openshift-image-registry/node-ca-kdsx4"
Jul 10 15:55:15.225635 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225491    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-systemd-units\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225635 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225535    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-run-netns\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225635 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225560    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-run-ovn-kubernetes\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225635 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225595    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/112868fd-c401-4f97-81a9-6a1355d9046b-ovnkube-script-lib\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225635 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225618    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76d52889-8bfd-412c-b4fe-3943f4e074a0-hosts-file\") pod \"node-resolver-htc8d\" (UID: \"76d52889-8bfd-412c-b4fe-3943f4e074a0\") " pod="openshift-dns/node-resolver-htc8d"
Jul 10 15:55:15.225904 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225641    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lxmh\" (UniqueName: \"kubernetes.io/projected/9fc05a30-9841-4627-a43e-5591964cce62-kube-api-access-9lxmh\") pod \"node-ca-kdsx4\" (UID: \"9fc05a30-9841-4627-a43e-5591964cce62\") " pod="openshift-image-registry/node-ca-kdsx4"
Jul 10 15:55:15.225904 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225681    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2764\" (UniqueName: \"kubernetes.io/projected/76d52889-8bfd-412c-b4fe-3943f4e074a0-kube-api-access-z2764\") pod \"node-resolver-htc8d\" (UID: \"76d52889-8bfd-412c-b4fe-3943f4e074a0\") " pod="openshift-dns/node-resolver-htc8d"
Jul 10 15:55:15.225904 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225709    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-var-lib-openvswitch\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225904 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225742    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-run-openvswitch\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225904 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225767    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-run-systemd\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225904 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225811    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-etc-openvswitch\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225904 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225821    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:15.225904 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225836    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-run-ovn\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225904 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225861    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-cni-netd\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.225904 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.225877    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:15.225904 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225884    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.226386 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225908    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/112868fd-c401-4f97-81a9-6a1355d9046b-ovnkube-config\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.226386 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jlbp\" (UniqueName: \"kubernetes.io/projected/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-kube-api-access-9jlbp\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:15.226386 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225960    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76d52889-8bfd-412c-b4fe-3943f4e074a0-tmp-dir\") pod \"node-resolver-htc8d\" (UID: \"76d52889-8bfd-412c-b4fe-3943f4e074a0\") " pod="openshift-dns/node-resolver-htc8d"
Jul 10 15:55:15.226386 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.225985    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/112868fd-c401-4f97-81a9-6a1355d9046b-ovn-node-metrics-cert\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.226386 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.226011    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-node-log\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.227362 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.227278    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 15:55:15.227531 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.227513    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 15:55:15.227602 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.227527    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.228298 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.228274    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 15:55:15.228826 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.228804    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wwtq4\""
Jul 10 15:55:15.228921 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.228829    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 15:55:15.229146 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.229128    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 15:55:15.229946 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.229758    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 15:55:15.229946 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.229827    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g52mv\""
Jul 10 15:55:15.230923 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.230860    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 15:55:15.231431 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.231391    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zkn26\""
Jul 10 15:55:15.248424 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.248381    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pnwls"
Jul 10 15:55:15.254704 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.254674    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pnwls"
Jul 10 15:55:15.315863 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.315831    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 15:55:15.327007 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.326981    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2np7h\" (UniqueName: \"kubernetes.io/projected/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-kube-api-access-2np7h\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.327177 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327018    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64hxk\" (UniqueName: \"kubernetes.io/projected/5f4e55bc-c837-4dcc-b170-128c1eb18064-kube-api-access-64hxk\") pod \"iptables-alerter-4czq2\" (UID: \"5f4e55bc-c837-4dcc-b170-128c1eb18064\") " pod="openshift-network-operator/iptables-alerter-4czq2"
Jul 10 15:55:15.327177 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327043    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-etc-selinux\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.327177 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327098    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-cnibin\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.327177 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327133    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-cnibin\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.327387 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327177    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-run\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.327387 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327224    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f4e55bc-c837-4dcc-b170-128c1eb18064-host-slash\") pod \"iptables-alerter-4czq2\" (UID: \"5f4e55bc-c837-4dcc-b170-128c1eb18064\") " pod="openshift-network-operator/iptables-alerter-4czq2"
Jul 10 15:55:15.327387 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327272    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5xk4\" (UniqueName: \"kubernetes.io/projected/f777717d-0890-49b7-9b23-cd37eee5f907-kube-api-access-h5xk4\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.327387 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327298    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-hostroot\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.327387 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327328    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/112868fd-c401-4f97-81a9-6a1355d9046b-ovnkube-script-lib\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.327387 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327357    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76d52889-8bfd-412c-b4fe-3943f4e074a0-hosts-file\") pod \"node-resolver-htc8d\" (UID: \"76d52889-8bfd-412c-b4fe-3943f4e074a0\") " pod="openshift-dns/node-resolver-htc8d"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327481    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lxmh\" (UniqueName: \"kubernetes.io/projected/9fc05a30-9841-4627-a43e-5591964cce62-kube-api-access-9lxmh\") pod \"node-ca-kdsx4\" (UID: \"9fc05a30-9841-4627-a43e-5591964cce62\") " pod="openshift-image-registry/node-ca-kdsx4"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327515    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6q4k\" (UniqueName: \"kubernetes.io/projected/e64eb88a-9c8f-4ec8-858e-7c82904ab121-kube-api-access-z6q4k\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327537    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2764\" (UniqueName: \"kubernetes.io/projected/76d52889-8bfd-412c-b4fe-3943f4e074a0-kube-api-access-z2764\") pod \"node-resolver-htc8d\" (UID: \"76d52889-8bfd-412c-b4fe-3943f4e074a0\") " pod="openshift-dns/node-resolver-htc8d"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327554    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-var-lib-openvswitch\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327568    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-systemd\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327590    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hwjp\" (UniqueName: \"kubernetes.io/projected/25e5764e-208b-4b28-99cf-411f4b417b83-kube-api-access-2hwjp\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-run-systemd\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327634    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-cni-netd\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327653    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jlbp\" (UniqueName: \"kubernetes.io/projected/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-kube-api-access-9jlbp\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327681    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e64eb88a-9c8f-4ec8-858e-7c82904ab121-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327701    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-sysconfig\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327483    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76d52889-8bfd-412c-b4fe-3943f4e074a0-hosts-file\") pod \"node-resolver-htc8d\" (UID: \"76d52889-8bfd-412c-b4fe-3943f4e074a0\") " pod="openshift-dns/node-resolver-htc8d"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327723    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-run-netns\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.327743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327741    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16d3e229-e008-438f-8f5a-13d9364e5b7b-kubelet-config\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-log-socket\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327786    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwmmp\" (UniqueName: \"kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp\") pod \"network-check-target-xbfbt\" (UID: \"25a66fdf-5eef-4fd3-b7b6-369254a11b3f\") " pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327803    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-kubernetes\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327823    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-host\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327849    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-modprobe-d\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-lib-modules\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-tmp\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327914    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-device-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327953    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-var-lib-openvswitch\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327970    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-run-systemd\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.327979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-sys-fs\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-multus-cni-dir\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328008    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/112868fd-c401-4f97-81a9-6a1355d9046b-ovnkube-script-lib\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328022    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-run-k8s-cni-cncf-io\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328011    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-cni-netd\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328043    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fc05a30-9841-4627-a43e-5591964cce62-host\") pod \"node-ca-kdsx4\" (UID: \"9fc05a30-9841-4627-a43e-5591964cce62\") " pod="openshift-image-registry/node-ca-kdsx4"
Jul 10 15:55:15.328380 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328069    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-sysctl-conf\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328096    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-var-lib-cni-multus\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328151    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-multus-conf-dir\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328164    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-log-socket\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328174    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-run-multus-certs\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-etc-kubernetes\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328217    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328204    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fc05a30-9841-4627-a43e-5591964cce62-host\") pod \"node-ca-kdsx4\" (UID: \"9fc05a30-9841-4627-a43e-5591964cce62\") " pod="openshift-image-registry/node-ca-kdsx4"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328253    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9fc05a30-9841-4627-a43e-5591964cce62-serviceca\") pod \"node-ca-kdsx4\" (UID: \"9fc05a30-9841-4627-a43e-5591964cce62\") " pod="openshift-image-registry/node-ca-kdsx4"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328289    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-sysctl-d\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.328329    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328381    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f4e55bc-c837-4dcc-b170-128c1eb18064-iptables-alerter-script\") pod \"iptables-alerter-4czq2\" (UID: \"5f4e55bc-c837-4dcc-b170-128c1eb18064\") " pod="openshift-network-operator/iptables-alerter-4czq2"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.328429    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs podName:232217f5-52c2-4bd8-b0f7-73f950e6c2a5 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:15.828389801 +0000 UTC m=+2.971923628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs") pod "network-metrics-daemon-lh5cf" (UID: "232217f5-52c2-4bd8-b0f7-73f950e6c2a5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328536    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-var-lib-cni-bin\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328576    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-systemd-units\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328597    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-run-netns\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328615    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-run-ovn-kubernetes\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329082 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328644    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-os-release\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328669    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-run-netns\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328709    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-systemd-units\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328750    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-var-lib-kubelet\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328759    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9fc05a30-9841-4627-a43e-5591964cce62-serviceca\") pod \"node-ca-kdsx4\" (UID: \"9fc05a30-9841-4627-a43e-5591964cce62\") " pod="openshift-image-registry/node-ca-kdsx4"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328765    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-run-ovn-kubernetes\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-socket-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328809    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-run-openvswitch\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328848    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9133bd56-dbfb-4b6a-8b50-d96c637055a3-konnectivity-ca\") pod \"konnectivity-agent-hw282\" (UID: \"9133bd56-dbfb-4b6a-8b50-d96c637055a3\") " pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-run-openvswitch\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328918    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-etc-openvswitch\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328943    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-run-ovn\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.328977    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329003    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/112868fd-c401-4f97-81a9-6a1355d9046b-ovnkube-config\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329025    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76d52889-8bfd-412c-b4fe-3943f4e074a0-tmp-dir\") pod \"node-resolver-htc8d\" (UID: \"76d52889-8bfd-412c-b4fe-3943f4e074a0\") " pod="openshift-dns/node-resolver-htc8d"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329026    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-etc-openvswitch\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.329714 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329026    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-run-ovn\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329062    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329070    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e64eb88a-9c8f-4ec8-858e-7c82904ab121-cni-binary-copy\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329095    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9133bd56-dbfb-4b6a-8b50-d96c637055a3-agent-certs\") pod \"konnectivity-agent-hw282\" (UID: \"9133bd56-dbfb-4b6a-8b50-d96c637055a3\") " pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329118    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-os-release\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329146    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/112868fd-c401-4f97-81a9-6a1355d9046b-ovn-node-metrics-cert\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329171    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329195    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16d3e229-e008-438f-8f5a-13d9364e5b7b-dbus\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329272    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-system-cni-dir\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329297    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-var-lib-kubelet\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-node-log\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329349    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-cni-bin\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329354    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76d52889-8bfd-412c-b4fe-3943f4e074a0-tmp-dir\") pod \"node-resolver-htc8d\" (UID: \"76d52889-8bfd-412c-b4fe-3943f4e074a0\") " pod="openshift-dns/node-resolver-htc8d"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329373    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrzxc\" (UniqueName: \"kubernetes.io/projected/112868fd-c401-4f97-81a9-6a1355d9046b-kube-api-access-mrzxc\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329399    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329477    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-cni-bin\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329503    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/112868fd-c401-4f97-81a9-6a1355d9046b-ovnkube-config\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330312 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329517    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/25e5764e-208b-4b28-99cf-411f4b417b83-cni-binary-copy\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329556    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-node-log\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329589    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/25e5764e-208b-4b28-99cf-411f4b417b83-multus-daemon-config\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-kubelet\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-slash\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329659    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/112868fd-c401-4f97-81a9-6a1355d9046b-env-overrides\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329695    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-kubelet\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329738    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/112868fd-c401-4f97-81a9-6a1355d9046b-host-slash\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329759    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329773    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e64eb88a-9c8f-4ec8-858e-7c82904ab121-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329807    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-sys\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329829    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-registration-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329857    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-multus-socket-dir-parent\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-system-cni-dir\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329899    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-tuned\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.330839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.329912    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/112868fd-c401-4f97-81a9-6a1355d9046b-env-overrides\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.333044 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.333022    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/112868fd-c401-4f97-81a9-6a1355d9046b-ovn-node-metrics-cert\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.342247 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.342222    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrzxc\" (UniqueName: \"kubernetes.io/projected/112868fd-c401-4f97-81a9-6a1355d9046b-kube-api-access-mrzxc\") pod \"ovnkube-node-qz2jw\" (UID: \"112868fd-c401-4f97-81a9-6a1355d9046b\") " pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.342386 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.342219    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2764\" (UniqueName: \"kubernetes.io/projected/76d52889-8bfd-412c-b4fe-3943f4e074a0-kube-api-access-z2764\") pod \"node-resolver-htc8d\" (UID: \"76d52889-8bfd-412c-b4fe-3943f4e074a0\") " pod="openshift-dns/node-resolver-htc8d"
Jul 10 15:55:15.342386 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.342348    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lxmh\" (UniqueName: \"kubernetes.io/projected/9fc05a30-9841-4627-a43e-5591964cce62-kube-api-access-9lxmh\") pod \"node-ca-kdsx4\" (UID: \"9fc05a30-9841-4627-a43e-5591964cce62\") " pod="openshift-image-registry/node-ca-kdsx4"
Jul 10 15:55:15.342796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.342776    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jlbp\" (UniqueName: \"kubernetes.io/projected/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-kube-api-access-9jlbp\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:15.430671 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430580    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-modprobe-d\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.430671 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430625    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-lib-modules\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.430671 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430651    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-tmp\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430678    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-device-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-sys-fs\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430730    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-multus-cni-dir\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430755    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-run-k8s-cni-cncf-io\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430772    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-modprobe-d\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430779    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-sysctl-conf\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430800    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-device-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430835    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-sys-fs\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430851    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-var-lib-cni-multus\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430777    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-lib-modules\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430889    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-multus-conf-dir\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430892    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-run-k8s-cni-cncf-io\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430912    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-sysctl-conf\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430921    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-run-multus-certs\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.430938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430929    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-var-lib-cni-multus\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430952    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-etc-kubernetes\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430970    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-multus-conf-dir\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.430971    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-multus-cni-dir\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431014    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-etc-kubernetes\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431050    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-sysctl-d\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431054    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-run-multus-certs\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431087    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f4e55bc-c837-4dcc-b170-128c1eb18064-iptables-alerter-script\") pod \"iptables-alerter-4czq2\" (UID: \"5f4e55bc-c837-4dcc-b170-128c1eb18064\") " pod="openshift-network-operator/iptables-alerter-4czq2"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431116    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-var-lib-cni-bin\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431142    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-os-release\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431147    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-sysctl-d\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431164    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-var-lib-kubelet\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431186    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-socket-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431210    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9133bd56-dbfb-4b6a-8b50-d96c637055a3-konnectivity-ca\") pod \"konnectivity-agent-hw282\" (UID: \"9133bd56-dbfb-4b6a-8b50-d96c637055a3\") " pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431227    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-os-release\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431232    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431253    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-var-lib-kubelet\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.431675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431205    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-var-lib-cni-bin\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e64eb88a-9c8f-4ec8-858e-7c82904ab121-cni-binary-copy\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431305    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9133bd56-dbfb-4b6a-8b50-d96c637055a3-agent-certs\") pod \"konnectivity-agent-hw282\" (UID: \"9133bd56-dbfb-4b6a-8b50-d96c637055a3\") " pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431322    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-socket-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431329    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-os-release\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431357    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431382    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16d3e229-e008-438f-8f5a-13d9364e5b7b-dbus\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431425    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-system-cni-dir\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431449    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-var-lib-kubelet\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431480    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431508    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/25e5764e-208b-4b28-99cf-411f4b417b83-cni-binary-copy\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/25e5764e-208b-4b28-99cf-411f4b417b83-multus-daemon-config\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e64eb88a-9c8f-4ec8-858e-7c82904ab121-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431600    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-sys\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431625    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-registration-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431650    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-multus-socket-dir-parent\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431674    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-system-cni-dir\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.432512 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-tuned\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431722    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2np7h\" (UniqueName: \"kubernetes.io/projected/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-kube-api-access-2np7h\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431747    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64hxk\" (UniqueName: \"kubernetes.io/projected/5f4e55bc-c837-4dcc-b170-128c1eb18064-kube-api-access-64hxk\") pod \"iptables-alerter-4czq2\" (UID: \"5f4e55bc-c837-4dcc-b170-128c1eb18064\") " pod="openshift-network-operator/iptables-alerter-4czq2"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431771    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-etc-selinux\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431795    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-cnibin\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431829    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-cnibin\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431830    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9133bd56-dbfb-4b6a-8b50-d96c637055a3-konnectivity-ca\") pod \"konnectivity-agent-hw282\" (UID: \"9133bd56-dbfb-4b6a-8b50-d96c637055a3\") " pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-run\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e64eb88a-9c8f-4ec8-858e-7c82904ab121-cni-binary-copy\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431878    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f4e55bc-c837-4dcc-b170-128c1eb18064-host-slash\") pod \"iptables-alerter-4czq2\" (UID: \"5f4e55bc-c837-4dcc-b170-128c1eb18064\") " pod="openshift-network-operator/iptables-alerter-4czq2"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431910    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5xk4\" (UniqueName: \"kubernetes.io/projected/f777717d-0890-49b7-9b23-cd37eee5f907-kube-api-access-h5xk4\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.431919    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431923    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-system-cni-dir\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-hostroot\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.431972    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret podName:16d3e229-e008-438f-8f5a-13d9364e5b7b nodeName:}" failed. No retries permitted until 2026-07-10 15:55:15.931954217 +0000 UTC m=+3.075488049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret") pod "global-pull-secret-syncer-l5nkc" (UID: "16d3e229-e008-438f-8f5a-13d9364e5b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431981    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-os-release\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431992    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6q4k\" (UniqueName: \"kubernetes.io/projected/e64eb88a-9c8f-4ec8-858e-7c82904ab121-kube-api-access-z6q4k\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.433365 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432035    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-systemd\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432059    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hwjp\" (UniqueName: \"kubernetes.io/projected/25e5764e-208b-4b28-99cf-411f4b417b83-kube-api-access-2hwjp\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432080    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-registration-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432098    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-etc-selinux\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432146    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432152    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-multus-socket-dir-parent\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432196    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-system-cni-dir\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432266    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16d3e229-e008-438f-8f5a-13d9364e5b7b-dbus\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432305    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-cnibin\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.431802    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f4e55bc-c837-4dcc-b170-128c1eb18064-iptables-alerter-script\") pod \"iptables-alerter-4czq2\" (UID: \"5f4e55bc-c837-4dcc-b170-128c1eb18064\") " pod="openshift-network-operator/iptables-alerter-4czq2"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432324    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f777717d-0890-49b7-9b23-cd37eee5f907-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432005    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-hostroot\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432374    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e64eb88a-9c8f-4ec8-858e-7c82904ab121-cnibin\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432396    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f4e55bc-c837-4dcc-b170-128c1eb18064-host-slash\") pod \"iptables-alerter-4czq2\" (UID: \"5f4e55bc-c837-4dcc-b170-128c1eb18064\") " pod="openshift-network-operator/iptables-alerter-4czq2"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432420    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-run\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432428    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-var-lib-kubelet\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432492    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-sys\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.434254 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432649    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-systemd\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e64eb88a-9c8f-4ec8-858e-7c82904ab121-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.432088    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e64eb88a-9c8f-4ec8-858e-7c82904ab121-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433056    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e64eb88a-9c8f-4ec8-858e-7c82904ab121-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433087    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-sysconfig\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-run-netns\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16d3e229-e008-438f-8f5a-13d9364e5b7b-kubelet-config\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmmp\" (UniqueName: \"kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp\") pod \"network-check-target-xbfbt\" (UID: \"25a66fdf-5eef-4fd3-b7b6-369254a11b3f\") " pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433191    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-kubernetes\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433201    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25e5764e-208b-4b28-99cf-411f4b417b83-host-run-netns\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433215    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-host\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433251    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-sysconfig\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433289    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-host\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433316    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/25e5764e-208b-4b28-99cf-411f4b417b83-multus-daemon-config\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433330    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-kubernetes\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433334    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16d3e229-e008-438f-8f5a-13d9364e5b7b-kubelet-config\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.433579    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-tmp\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.435115 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.434327    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9133bd56-dbfb-4b6a-8b50-d96c637055a3-agent-certs\") pod \"konnectivity-agent-hw282\" (UID: \"9133bd56-dbfb-4b6a-8b50-d96c637055a3\") " pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:15.436024 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.434508    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-etc-tuned\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.437043 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.437023    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/25e5764e-208b-4b28-99cf-411f4b417b83-cni-binary-copy\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.446689 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.446662    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64hxk\" (UniqueName: \"kubernetes.io/projected/5f4e55bc-c837-4dcc-b170-128c1eb18064-kube-api-access-64hxk\") pod \"iptables-alerter-4czq2\" (UID: \"5f4e55bc-c837-4dcc-b170-128c1eb18064\") " pod="openshift-network-operator/iptables-alerter-4czq2"
Jul 10 15:55:15.446872 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.446854    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5xk4\" (UniqueName: \"kubernetes.io/projected/f777717d-0890-49b7-9b23-cd37eee5f907-kube-api-access-h5xk4\") pod \"aws-ebs-csi-driver-node-kcvsp\" (UID: \"f777717d-0890-49b7-9b23-cd37eee5f907\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.447207 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.447190    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2np7h\" (UniqueName: \"kubernetes.io/projected/15a3edc0-07c5-4f8f-83d9-049c5d274bdc-kube-api-access-2np7h\") pod \"tuned-k5759\" (UID: \"15a3edc0-07c5-4f8f-83d9-049c5d274bdc\") " pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.451614 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.451592    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 15:55:15.451614 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.451618    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 15:55:15.451807 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.451632    2567 projected.go:196] Error preparing data for projected volume kube-api-access-qwmmp for pod openshift-network-diagnostics/network-check-target-xbfbt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:15.451807 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.451719    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp podName:25a66fdf-5eef-4fd3-b7b6-369254a11b3f nodeName:}" failed. No retries permitted until 2026-07-10 15:55:15.95169886 +0000 UTC m=+3.095232686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qwmmp" (UniqueName: "kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp") pod "network-check-target-xbfbt" (UID: "25a66fdf-5eef-4fd3-b7b6-369254a11b3f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:15.453386 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.453365    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hwjp\" (UniqueName: \"kubernetes.io/projected/25e5764e-208b-4b28-99cf-411f4b417b83-kube-api-access-2hwjp\") pod \"multus-frcsg\" (UID: \"25e5764e-208b-4b28-99cf-411f4b417b83\") " pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.457720 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.457694    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6q4k\" (UniqueName: \"kubernetes.io/projected/e64eb88a-9c8f-4ec8-858e-7c82904ab121-kube-api-access-z6q4k\") pod \"multus-additional-cni-plugins-lwpkp\" (UID: \"e64eb88a-9c8f-4ec8-858e-7c82904ab121\") " pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.529292 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.529252    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:15.539077 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.539053    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-htc8d"
Jul 10 15:55:15.545802 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.545766    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kdsx4"
Jul 10 15:55:15.557601 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.557574    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lwpkp"
Jul 10 15:55:15.564469 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.564447    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:15.573270 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.573243    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4czq2"
Jul 10 15:55:15.581110 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.581077    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k5759"
Jul 10 15:55:15.587770 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.587749    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp"
Jul 10 15:55:15.592337 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.592311    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-frcsg"
Jul 10 15:55:15.836613 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.836535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:15.836749 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.836661    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:15.836749 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.836724    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs podName:232217f5-52c2-4bd8-b0f7-73f950e6c2a5 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:16.836709755 +0000 UTC m=+3.980243571 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs") pod "network-metrics-daemon-lh5cf" (UID: "232217f5-52c2-4bd8-b0f7-73f950e6c2a5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:15.936949 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:15.936908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:15.937120 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.937042    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:15.937120 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:15.937101    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret podName:16d3e229-e008-438f-8f5a-13d9364e5b7b nodeName:}" failed. No retries permitted until 2026-07-10 15:55:16.93708249 +0000 UTC m=+4.080616307 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret") pod "global-pull-secret-syncer-l5nkc" (UID: "16d3e229-e008-438f-8f5a-13d9364e5b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:16.037543 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.037495    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmmp\" (UniqueName: \"kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp\") pod \"network-check-target-xbfbt\" (UID: \"25a66fdf-5eef-4fd3-b7b6-369254a11b3f\") " pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:16.037732 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:16.037684    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 15:55:16.037732 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:16.037709    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 15:55:16.037732 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:16.037723    2567 projected.go:196] Error preparing data for projected volume kube-api-access-qwmmp for pod openshift-network-diagnostics/network-check-target-xbfbt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:16.037852 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:16.037777    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp podName:25a66fdf-5eef-4fd3-b7b6-369254a11b3f nodeName:}" failed. No retries permitted until 2026-07-10 15:55:17.037762971 +0000 UTC m=+4.181296791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwmmp" (UniqueName: "kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp") pod "network-check-target-xbfbt" (UID: "25a66fdf-5eef-4fd3-b7b6-369254a11b3f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:16.196099 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:16.196056    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d52889_8bfd_412c_b4fe_3943f4e074a0.slice/crio-987eba5f73e5856df92a977e435e3d70082c184e406f089c948bfd7d8b34e4c5 WatchSource:0}: Error finding container 987eba5f73e5856df92a977e435e3d70082c184e406f089c948bfd7d8b34e4c5: Status 404 returned error can't find the container with id 987eba5f73e5856df92a977e435e3d70082c184e406f089c948bfd7d8b34e4c5
Jul 10 15:55:16.197873 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:16.197845    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e5764e_208b_4b28_99cf_411f4b417b83.slice/crio-c449a0ee3cf968274569c119a44fc44ffe40333622f67891b9eecaf407991d87 WatchSource:0}: Error finding container c449a0ee3cf968274569c119a44fc44ffe40333622f67891b9eecaf407991d87: Status 404 returned error can't find the container with id c449a0ee3cf968274569c119a44fc44ffe40333622f67891b9eecaf407991d87
Jul 10 15:55:16.198722 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:16.198668    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode64eb88a_9c8f_4ec8_858e_7c82904ab121.slice/crio-a4a85c22a8a3eeffbd29653443b84cdd21b1edbfcdd0b71bbbf24b16c6f06fe8 WatchSource:0}: Error finding container a4a85c22a8a3eeffbd29653443b84cdd21b1edbfcdd0b71bbbf24b16c6f06fe8: Status 404 returned error can't find the container with id a4a85c22a8a3eeffbd29653443b84cdd21b1edbfcdd0b71bbbf24b16c6f06fe8
Jul 10 15:55:16.199811 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:16.199785    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf777717d_0890_49b7_9b23_cd37eee5f907.slice/crio-ba9b8ddf1343c061ce77ae4f1b1f3937368417db00fecc91598ef53e1a2ed6f5 WatchSource:0}: Error finding container ba9b8ddf1343c061ce77ae4f1b1f3937368417db00fecc91598ef53e1a2ed6f5: Status 404 returned error can't find the container with id ba9b8ddf1343c061ce77ae4f1b1f3937368417db00fecc91598ef53e1a2ed6f5
Jul 10 15:55:16.203277 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:16.203256    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f4e55bc_c837_4dcc_b170_128c1eb18064.slice/crio-25311b5986b28cdd8d6753ad4485784dcc730e9a3301b011cc5704c2809ffebb WatchSource:0}: Error finding container 25311b5986b28cdd8d6753ad4485784dcc730e9a3301b011cc5704c2809ffebb: Status 404 returned error can't find the container with id 25311b5986b28cdd8d6753ad4485784dcc730e9a3301b011cc5704c2809ffebb
Jul 10 15:55:16.204274 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:16.204248    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a3edc0_07c5_4f8f_83d9_049c5d274bdc.slice/crio-ed7f6cac1c2646fb5d4efe61fc5d2d6f313af43f7d5ceadceb42fb2a5552f358 WatchSource:0}: Error finding container ed7f6cac1c2646fb5d4efe61fc5d2d6f313af43f7d5ceadceb42fb2a5552f358: Status 404 returned error can't find the container with id ed7f6cac1c2646fb5d4efe61fc5d2d6f313af43f7d5ceadceb42fb2a5552f358
Jul 10 15:55:16.204888 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:16.204852    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9133bd56_dbfb_4b6a_8b50_d96c637055a3.slice/crio-5bb0aab1653f5e062f5f11bbbdaff97aa4fb85fe6b6ecb66e4c66f429e28effb WatchSource:0}: Error finding container 5bb0aab1653f5e062f5f11bbbdaff97aa4fb85fe6b6ecb66e4c66f429e28effb: Status 404 returned error can't find the container with id 5bb0aab1653f5e062f5f11bbbdaff97aa4fb85fe6b6ecb66e4c66f429e28effb
Jul 10 15:55:16.206234 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:16.206081    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod112868fd_c401_4f97_81a9_6a1355d9046b.slice/crio-abfac9596573c892463aa4815c049159d81d4df9d94da78118b5378756787bb9 WatchSource:0}: Error finding container abfac9596573c892463aa4815c049159d81d4df9d94da78118b5378756787bb9: Status 404 returned error can't find the container with id abfac9596573c892463aa4815c049159d81d4df9d94da78118b5378756787bb9
Jul 10 15:55:16.207708 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:16.207476    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fc05a30_9841_4627_a43e_5591964cce62.slice/crio-b88ff2a3314c7b588d7ef3212f2c26e533e1b977118e4a3ccae9275f7782d263 WatchSource:0}: Error finding container b88ff2a3314c7b588d7ef3212f2c26e533e1b977118e4a3ccae9275f7782d263: Status 404 returned error can't find the container with id b88ff2a3314c7b588d7ef3212f2c26e533e1b977118e4a3ccae9275f7782d263
Jul 10 15:55:16.257560 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.257483    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 15:50:15 +0000 UTC" deadline="2028-03-31 15:30:53.506087738 +0000 UTC"
Jul 10 15:55:16.257560 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.257547    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15119h35m37.248543467s"
Jul 10 15:55:16.408633 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.408444    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-156.ec2.internal" event={"ID":"6a91c7d5b3d2028299be24baf3606bd6","Type":"ContainerStarted","Data":"afa2971224936d20018db6cadb33c71031f7eef66913c87415bad2dd0dafebf2"}
Jul 10 15:55:16.409728 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.409699    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kdsx4" event={"ID":"9fc05a30-9841-4627-a43e-5591964cce62","Type":"ContainerStarted","Data":"b88ff2a3314c7b588d7ef3212f2c26e533e1b977118e4a3ccae9275f7782d263"}
Jul 10 15:55:16.410935 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.410895    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw" event={"ID":"112868fd-c401-4f97-81a9-6a1355d9046b","Type":"ContainerStarted","Data":"abfac9596573c892463aa4815c049159d81d4df9d94da78118b5378756787bb9"}
Jul 10 15:55:16.411910 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.411888    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hw282" event={"ID":"9133bd56-dbfb-4b6a-8b50-d96c637055a3","Type":"ContainerStarted","Data":"5bb0aab1653f5e062f5f11bbbdaff97aa4fb85fe6b6ecb66e4c66f429e28effb"}
Jul 10 15:55:16.412898 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.412875    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp" event={"ID":"f777717d-0890-49b7-9b23-cd37eee5f907","Type":"ContainerStarted","Data":"ba9b8ddf1343c061ce77ae4f1b1f3937368417db00fecc91598ef53e1a2ed6f5"}
Jul 10 15:55:16.413782 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.413752    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwpkp" event={"ID":"e64eb88a-9c8f-4ec8-858e-7c82904ab121","Type":"ContainerStarted","Data":"a4a85c22a8a3eeffbd29653443b84cdd21b1edbfcdd0b71bbbf24b16c6f06fe8"}
Jul 10 15:55:16.415144 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.415115    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal" event={"ID":"23961ef0df5a0f6fd14429a033775076","Type":"ContainerStarted","Data":"331c8a0bb671e08a17c0d0eb9f2def1e85cd2108c1a8fb78f7aec2e9cda1f097"}
Jul 10 15:55:16.416250 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.416230    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k5759" event={"ID":"15a3edc0-07c5-4f8f-83d9-049c5d274bdc","Type":"ContainerStarted","Data":"ed7f6cac1c2646fb5d4efe61fc5d2d6f313af43f7d5ceadceb42fb2a5552f358"}
Jul 10 15:55:16.417272 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.417251    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4czq2" event={"ID":"5f4e55bc-c837-4dcc-b170-128c1eb18064","Type":"ContainerStarted","Data":"25311b5986b28cdd8d6753ad4485784dcc730e9a3301b011cc5704c2809ffebb"}
Jul 10 15:55:16.418448 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.418426    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-frcsg" event={"ID":"25e5764e-208b-4b28-99cf-411f4b417b83","Type":"ContainerStarted","Data":"c449a0ee3cf968274569c119a44fc44ffe40333622f67891b9eecaf407991d87"}
Jul 10 15:55:16.419492 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.419472    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-htc8d" event={"ID":"76d52889-8bfd-412c-b4fe-3943f4e074a0","Type":"ContainerStarted","Data":"987eba5f73e5856df92a977e435e3d70082c184e406f089c948bfd7d8b34e4c5"}
Jul 10 15:55:16.424447 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.424384    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-156.ec2.internal" podStartSLOduration=2.424371252 podStartE2EDuration="2.424371252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:16.42342483 +0000 UTC m=+3.566958670" watchObservedRunningTime="2026-07-10 15:55:16.424371252 +0000 UTC m=+3.567905090"
Jul 10 15:55:16.843775 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.843687    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:16.843948 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:16.843898    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:16.844012 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:16.843965    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs podName:232217f5-52c2-4bd8-b0f7-73f950e6c2a5 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:18.843944317 +0000 UTC m=+5.987478138 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs") pod "network-metrics-daemon-lh5cf" (UID: "232217f5-52c2-4bd8-b0f7-73f950e6c2a5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:16.945133 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:16.944504    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:16.945133 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:16.944716    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:16.945133 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:16.944781    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret podName:16d3e229-e008-438f-8f5a-13d9364e5b7b nodeName:}" failed. No retries permitted until 2026-07-10 15:55:18.944761328 +0000 UTC m=+6.088295165 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret") pod "global-pull-secret-syncer-l5nkc" (UID: "16d3e229-e008-438f-8f5a-13d9364e5b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:17.044970 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:17.044926    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmmp\" (UniqueName: \"kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp\") pod \"network-check-target-xbfbt\" (UID: \"25a66fdf-5eef-4fd3-b7b6-369254a11b3f\") " pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:17.045144 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:17.045130    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 15:55:17.045223 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:17.045150    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 15:55:17.045223 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:17.045163    2567 projected.go:196] Error preparing data for projected volume kube-api-access-qwmmp for pod openshift-network-diagnostics/network-check-target-xbfbt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:17.045319 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:17.045222    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp podName:25a66fdf-5eef-4fd3-b7b6-369254a11b3f nodeName:}" failed. No retries permitted until 2026-07-10 15:55:19.045203319 +0000 UTC m=+6.188737152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwmmp" (UniqueName: "kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp") pod "network-check-target-xbfbt" (UID: "25a66fdf-5eef-4fd3-b7b6-369254a11b3f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:17.259699 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:17.259601    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 15:50:15 +0000 UTC" deadline="2028-01-17 14:02:24.651366895 +0000 UTC"
Jul 10 15:55:17.259699 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:17.259646    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13342h7m7.391725345s"
Jul 10 15:55:17.399652 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:17.399552    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:17.399652 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:17.399673    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:17.400128 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:17.400100    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:17.400224 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:17.400205    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:17.400484 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:17.400309    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:17.400484 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:17.400398    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:17.448476 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:17.448437    2567 generic.go:356] "Generic (PLEG): container finished" podID="23961ef0df5a0f6fd14429a033775076" containerID="331c8a0bb671e08a17c0d0eb9f2def1e85cd2108c1a8fb78f7aec2e9cda1f097" exitCode=0
Jul 10 15:55:17.448655 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:17.448602    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal" event={"ID":"23961ef0df5a0f6fd14429a033775076","Type":"ContainerDied","Data":"331c8a0bb671e08a17c0d0eb9f2def1e85cd2108c1a8fb78f7aec2e9cda1f097"}
Jul 10 15:55:18.475961 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:18.475915    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal" event={"ID":"23961ef0df5a0f6fd14429a033775076","Type":"ContainerStarted","Data":"c5545a050c3a891b60a4cee5cac6ee3697c90e6d95e13242e48180e00ad21db8"}
Jul 10 15:55:18.869673 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:18.869584    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:18.869837 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:18.869793    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:18.869902 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:18.869860    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs podName:232217f5-52c2-4bd8-b0f7-73f950e6c2a5 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:22.869841113 +0000 UTC m=+10.013374936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs") pod "network-metrics-daemon-lh5cf" (UID: "232217f5-52c2-4bd8-b0f7-73f950e6c2a5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:18.970131 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:18.970073    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:18.970334 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:18.970232    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:18.970334 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:18.970298    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret podName:16d3e229-e008-438f-8f5a-13d9364e5b7b nodeName:}" failed. No retries permitted until 2026-07-10 15:55:22.970277091 +0000 UTC m=+10.113810914 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret") pod "global-pull-secret-syncer-l5nkc" (UID: "16d3e229-e008-438f-8f5a-13d9364e5b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:19.073239 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:19.071062    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmmp\" (UniqueName: \"kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp\") pod \"network-check-target-xbfbt\" (UID: \"25a66fdf-5eef-4fd3-b7b6-369254a11b3f\") " pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:19.073239 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:19.071288    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 15:55:19.073239 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:19.071311    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 15:55:19.073239 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:19.071326    2567 projected.go:196] Error preparing data for projected volume kube-api-access-qwmmp for pod openshift-network-diagnostics/network-check-target-xbfbt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:19.073239 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:19.071393    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp podName:25a66fdf-5eef-4fd3-b7b6-369254a11b3f nodeName:}" failed. No retries permitted until 2026-07-10 15:55:23.071373682 +0000 UTC m=+10.214907523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwmmp" (UniqueName: "kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp") pod "network-check-target-xbfbt" (UID: "25a66fdf-5eef-4fd3-b7b6-369254a11b3f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:19.400598 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:19.400564    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:19.400782 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:19.400698    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:19.400782 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:19.400563    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:19.400911 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:19.400867    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:19.400965 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:19.400910    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:19.401020 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:19.401005    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:21.399786 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:21.399312    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:21.399786 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:21.399316    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:21.399786 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:21.399447    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:21.399786 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:21.399460    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:21.399786 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:21.399571    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:21.399786 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:21.399712    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:22.906662 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:22.906619    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:22.907122 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:22.906815    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:22.907122 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:22.906879    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs podName:232217f5-52c2-4bd8-b0f7-73f950e6c2a5 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:30.906860205 +0000 UTC m=+18.050394045 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs") pod "network-metrics-daemon-lh5cf" (UID: "232217f5-52c2-4bd8-b0f7-73f950e6c2a5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:23.007299 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:23.007258    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:23.007498 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:23.007431    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:23.007539 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:23.007502    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret podName:16d3e229-e008-438f-8f5a-13d9364e5b7b nodeName:}" failed. No retries permitted until 2026-07-10 15:55:31.00748379 +0000 UTC m=+18.151017610 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret") pod "global-pull-secret-syncer-l5nkc" (UID: "16d3e229-e008-438f-8f5a-13d9364e5b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:23.107676 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:23.107637    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmmp\" (UniqueName: \"kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp\") pod \"network-check-target-xbfbt\" (UID: \"25a66fdf-5eef-4fd3-b7b6-369254a11b3f\") " pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:23.107887 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:23.107868    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 15:55:23.107949 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:23.107894    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 15:55:23.107949 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:23.107908    2567 projected.go:196] Error preparing data for projected volume kube-api-access-qwmmp for pod openshift-network-diagnostics/network-check-target-xbfbt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:23.108058 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:23.107968    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp podName:25a66fdf-5eef-4fd3-b7b6-369254a11b3f nodeName:}" failed. No retries permitted until 2026-07-10 15:55:31.107948836 +0000 UTC m=+18.251482660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwmmp" (UniqueName: "kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp") pod "network-check-target-xbfbt" (UID: "25a66fdf-5eef-4fd3-b7b6-369254a11b3f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:23.400444 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:23.400015    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:23.400444 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:23.400124    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:23.400444 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:23.400141    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:23.400444 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:23.400214    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:23.400444 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:23.400266    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:23.400444 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:23.400318    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:25.399932 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:25.399752    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:25.399932 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:25.399876    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:25.399932 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:25.399760    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:25.400607 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:25.400014    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:25.400607 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:25.400096    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:25.400607 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:25.400441    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:27.399093 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:27.398991    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:27.399093 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:27.399016    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:27.399623 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:27.398992    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:27.399623 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:27.399133    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:27.399623 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:27.399203    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:27.399623 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:27.399256    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:29.399659 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:29.399617    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:29.400144 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:29.399623    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:29.400144 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:29.399761    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:29.400144 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:29.399623    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:29.400144 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:29.399835    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:29.400144 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:29.399913    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:30.967356 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:30.967314    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:30.967886 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:30.967481    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:30.967886 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:30.967542    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs podName:232217f5-52c2-4bd8-b0f7-73f950e6c2a5 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:46.96752674 +0000 UTC m=+34.111060557 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs") pod "network-metrics-daemon-lh5cf" (UID: "232217f5-52c2-4bd8-b0f7-73f950e6c2a5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:31.067923 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:31.067882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:31.068077 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:31.068051    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:31.068137 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:31.068121    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret podName:16d3e229-e008-438f-8f5a-13d9364e5b7b nodeName:}" failed. No retries permitted until 2026-07-10 15:55:47.068104394 +0000 UTC m=+34.211638213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret") pod "global-pull-secret-syncer-l5nkc" (UID: "16d3e229-e008-438f-8f5a-13d9364e5b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:31.169279 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:31.169228    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmmp\" (UniqueName: \"kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp\") pod \"network-check-target-xbfbt\" (UID: \"25a66fdf-5eef-4fd3-b7b6-369254a11b3f\") " pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:31.169476 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:31.169436    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 15:55:31.169476 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:31.169459    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 15:55:31.169476 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:31.169476    2567 projected.go:196] Error preparing data for projected volume kube-api-access-qwmmp for pod openshift-network-diagnostics/network-check-target-xbfbt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:31.169670 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:31.169537    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp podName:25a66fdf-5eef-4fd3-b7b6-369254a11b3f nodeName:}" failed. No retries permitted until 2026-07-10 15:55:47.169522115 +0000 UTC m=+34.313055933 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwmmp" (UniqueName: "kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp") pod "network-check-target-xbfbt" (UID: "25a66fdf-5eef-4fd3-b7b6-369254a11b3f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:31.399711 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:31.399619    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:31.399866 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:31.399738    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:31.399866 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:31.399757    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:31.399866 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:31.399819    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:31.399866 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:31.399842    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:31.400090 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:31.399938    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:33.400340 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:33.400303    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:33.400803 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:33.400388    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:33.400803 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:33.400455    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:33.400803 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:33.400534    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:33.400803 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:33.400534    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:33.400803 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:33.400631    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:35.399764 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:35.399726    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:35.399764 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:35.399741    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:35.400253 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:35.399744    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:35.400253 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:35.399850    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:35.400253 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:35.399917    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:35.400253 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:35.400002    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:36.515488 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.515274    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp" event={"ID":"f777717d-0890-49b7-9b23-cd37eee5f907","Type":"ContainerStarted","Data":"2751aed0821a95c34bff8842f8bb75e7b4168fe4d9b29531a12cd6ee3779f7dc"}
Jul 10 15:55:36.516891 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.516869    2567 generic.go:356] "Generic (PLEG): container finished" podID="e64eb88a-9c8f-4ec8-858e-7c82904ab121" containerID="7a55c0060f4b26bb5608d5bf4ce2c6c1e9c6d474317d76de9e525e925b89da98" exitCode=0
Jul 10 15:55:36.516952 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.516943    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwpkp" event={"ID":"e64eb88a-9c8f-4ec8-858e-7c82904ab121","Type":"ContainerDied","Data":"7a55c0060f4b26bb5608d5bf4ce2c6c1e9c6d474317d76de9e525e925b89da98"}
Jul 10 15:55:36.518590 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.518561    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k5759" event={"ID":"15a3edc0-07c5-4f8f-83d9-049c5d274bdc","Type":"ContainerStarted","Data":"7bb69dfc5f9d3f949b46f7d123de51a6c59037df1f22f80bfa0c1d6d90273350"}
Jul 10 15:55:36.520250 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.520224    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-frcsg" event={"ID":"25e5764e-208b-4b28-99cf-411f4b417b83","Type":"ContainerStarted","Data":"b6adec4f99e5f418de5fd5c9ad4e806050e2124513911ad28ab7f2a7d6ea66cd"}
Jul 10 15:55:36.521640 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.521617    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-htc8d" event={"ID":"76d52889-8bfd-412c-b4fe-3943f4e074a0","Type":"ContainerStarted","Data":"40641f90dda79e29dabf4be4dc5df2e1c1869a7b5815c3a30274446b5534ad1c"}
Jul 10 15:55:36.523125 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.523098    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kdsx4" event={"ID":"9fc05a30-9841-4627-a43e-5591964cce62","Type":"ContainerStarted","Data":"721ec44e531dbdf38f0e9909d65b0890bb0b373e2acf61102742cbadfa6157ae"}
Jul 10 15:55:36.524548 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.524529    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw" event={"ID":"112868fd-c401-4f97-81a9-6a1355d9046b","Type":"ContainerStarted","Data":"4ba453bab5f4f5bc8bdb2e51193bc1a78772c7a14a8febd101ac9ce48b4da613"}
Jul 10 15:55:36.525938 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.525920    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hw282" event={"ID":"9133bd56-dbfb-4b6a-8b50-d96c637055a3","Type":"ContainerStarted","Data":"d01f19f087ed5a04bfdf2f442e53fa28b01a87f7f351792ae27a2931627e84b4"}
Jul 10 15:55:36.544003 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.543947    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-156.ec2.internal" podStartSLOduration=22.54393248 podStartE2EDuration="22.54393248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:18.49598711 +0000 UTC m=+5.639520952" watchObservedRunningTime="2026-07-10 15:55:36.54393248 +0000 UTC m=+23.687466319"
Jul 10 15:55:36.564265 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.564212    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-htc8d" podStartSLOduration=4.417519735 podStartE2EDuration="23.564191501s" totalImagesPullingTime="19.146671766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:36.563247054 +0000 UTC m=+23.706780894" watchObservedRunningTime="2026-07-10 15:55:36.564191501 +0000 UTC m=+23.707725342"
Jul 10 15:55:36.602551 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.602355    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kdsx4" podStartSLOduration=4.467545139 podStartE2EDuration="23.602339606s" totalImagesPullingTime="19.134794467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:36.580555869 +0000 UTC m=+23.724089710" watchObservedRunningTime="2026-07-10 15:55:36.602339606 +0000 UTC m=+23.745873500"
Jul 10 15:55:36.602826 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.602804    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hw282" podStartSLOduration=12.631674475 podStartE2EDuration="23.602797546s" totalImagesPullingTime="10.971123071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:36.602181036 +0000 UTC m=+23.745714876" watchObservedRunningTime="2026-07-10 15:55:36.602797546 +0000 UTC m=+23.746331384"
Jul 10 15:55:36.709865 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.709813    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-k5759" podStartSLOduration=4.487538192 podStartE2EDuration="23.709795497s" totalImagesPullingTime="19.222257305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:36.66243617 +0000 UTC m=+23.805970006" watchObservedRunningTime="2026-07-10 15:55:36.709795497 +0000 UTC m=+23.853329330"
Jul 10 15:55:36.710389 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.710367    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-frcsg" podStartSLOduration=4.44495743 podStartE2EDuration="23.710358642s" totalImagesPullingTime="19.265401212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:36.709588997 +0000 UTC m=+23.853122836" watchObservedRunningTime="2026-07-10 15:55:36.710358642 +0000 UTC m=+23.853892477"
Jul 10 15:55:36.878042 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:36.878012    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 15:55:37.195288 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.195252    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:37.196001 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.195969    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:37.290806 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.290578    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T15:55:36.87803599Z","UUID":"07b5a678-9c5d-4737-96d2-f86dda957585","Handler":null,"Name":"","Endpoint":""}
Jul 10 15:55:37.292394 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.292370    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 15:55:37.292555 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.292433    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 15:55:37.399129 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.399087    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:37.399288 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:37.399224    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:37.399577 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.399559    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:37.399663 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:37.399645    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:37.399743 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.399720    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:37.399872 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:37.399808    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:37.529883 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.529843    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4czq2" event={"ID":"5f4e55bc-c837-4dcc-b170-128c1eb18064","Type":"ContainerStarted","Data":"154910584d2655d433b5c40a2684f98047ef81360efe8f018bc0b091a66c9b2a"}
Jul 10 15:55:37.533394 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.533361    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw" event={"ID":"112868fd-c401-4f97-81a9-6a1355d9046b","Type":"ContainerStarted","Data":"a6846342883381a9cdca053cde9f16d59c159be82debcd7e931a0248f059a833"}
Jul 10 15:55:37.533576 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.533473    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw" event={"ID":"112868fd-c401-4f97-81a9-6a1355d9046b","Type":"ContainerStarted","Data":"b2f24e10cdc6c37ba706381cc92f9df287bb94b2a6690aa806f738c07f37ef0f"}
Jul 10 15:55:37.533576 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.533526    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw" event={"ID":"112868fd-c401-4f97-81a9-6a1355d9046b","Type":"ContainerStarted","Data":"79a4cf429e522554bdd8cd2f08449d85aba85fc9e2a6c49deb4168f931d0df05"}
Jul 10 15:55:37.533576 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.533572    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw" event={"ID":"112868fd-c401-4f97-81a9-6a1355d9046b","Type":"ContainerStarted","Data":"d481ef5951ee31cb13317501a67837be958982f1c1a0e10bdf35ea186168802c"}
Jul 10 15:55:37.533739 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.533608    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw" event={"ID":"112868fd-c401-4f97-81a9-6a1355d9046b","Type":"ContainerStarted","Data":"fe09420e91c0506f76711a61d85942ad51c2a6fbbde85ef54eb77d4a637d9d19"}
Jul 10 15:55:37.535928 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.535899    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp" event={"ID":"f777717d-0890-49b7-9b23-cd37eee5f907","Type":"ContainerStarted","Data":"e7f10b36a12680a831b72e3b597d85a7f7dcd1b2e75c4feea3706c46fce15319"}
Jul 10 15:55:37.549428 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:37.549303    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4czq2" podStartSLOduration=5.403920077 podStartE2EDuration="24.54928049s" totalImagesPullingTime="19.145360413s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:37.549141408 +0000 UTC m=+24.692675247" watchObservedRunningTime="2026-07-10 15:55:37.54928049 +0000 UTC m=+24.692814333"
Jul 10 15:55:38.540777 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:38.540741    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp" event={"ID":"f777717d-0890-49b7-9b23-cd37eee5f907","Type":"ContainerStarted","Data":"eb47f2b8f96abb7c7a60c5d720ee92862b071903728bc28af08c46aebe34622b"}
Jul 10 15:55:38.540777 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:38.540775    2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 10 15:55:38.563208 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:38.563154    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcvsp" podStartSLOduration=4.170536621 podStartE2EDuration="25.563136615s" totalImagesPullingTime="21.392599994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:38.563064827 +0000 UTC m=+25.706598667" watchObservedRunningTime="2026-07-10 15:55:38.563136615 +0000 UTC m=+25.706670455"
Jul 10 15:55:39.403850 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:39.403809    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:39.404037 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:39.403822    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:39.404037 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:39.403938    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:39.404037 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:39.403980    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:39.404037 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:39.403830    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:39.404357 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:39.404131    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:41.399375 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:41.399195    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:41.400079 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:41.399196    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:41.400079 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:41.399469    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:41.400079 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:41.399207    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:41.400079 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:41.399554    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:41.400079 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:41.399635    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:41.550102 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:41.550066    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw" event={"ID":"112868fd-c401-4f97-81a9-6a1355d9046b","Type":"ContainerStarted","Data":"f6879a524042779e2599cfd4aa5fe32757216db0e53a537b41020d3d5ceddb85"}
Jul 10 15:55:41.551649 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:41.551624    2567 generic.go:356] "Generic (PLEG): container finished" podID="e64eb88a-9c8f-4ec8-858e-7c82904ab121" containerID="939357ef19fa364375aca2db823dc0989dd1fb9513288e483a9c417eba8d5330" exitCode=0
Jul 10 15:55:41.551751 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:41.551654    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwpkp" event={"ID":"e64eb88a-9c8f-4ec8-858e-7c82904ab121","Type":"ContainerDied","Data":"939357ef19fa364375aca2db823dc0989dd1fb9513288e483a9c417eba8d5330"}
Jul 10 15:55:43.399951 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:43.399729    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:43.400591 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:43.399782    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:43.400591 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:43.400053    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:43.400591 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:43.399816    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:43.400591 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:43.400134    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:43.400591 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:43.400188    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:43.560134 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:43.560094    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw" event={"ID":"112868fd-c401-4f97-81a9-6a1355d9046b","Type":"ContainerStarted","Data":"c5588fa2347f5461f87e81639cfbca7f2bdceaeefba781962fb5aa6236645972"}
Jul 10 15:55:43.560349 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:43.560331    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:43.562598 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:43.562567    2567 generic.go:356] "Generic (PLEG): container finished" podID="e64eb88a-9c8f-4ec8-858e-7c82904ab121" containerID="d1e6abf5fe5dfce39b1fcda0a717a073ec0df16a6daa3d390b1c18c1a1183b0d" exitCode=0
Jul 10 15:55:43.562723 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:43.562623    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwpkp" event={"ID":"e64eb88a-9c8f-4ec8-858e-7c82904ab121","Type":"ContainerDied","Data":"d1e6abf5fe5dfce39b1fcda0a717a073ec0df16a6daa3d390b1c18c1a1183b0d"}
Jul 10 15:55:43.577909 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:43.577881    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:43.599195 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:43.599143    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw" podStartSLOduration=10.607642592 podStartE2EDuration="30.599127473s" totalImagesPullingTime="19.991484881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:43.597521892 +0000 UTC m=+30.741055732" watchObservedRunningTime="2026-07-10 15:55:43.599127473 +0000 UTC m=+30.742661313"
Jul 10 15:55:44.564957 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:44.564928    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:44.565389 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:44.565024    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:44.574350 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:44.574319    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:44.574518 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:44.574491    2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 10 15:55:44.575161 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:44.575139    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hw282"
Jul 10 15:55:44.581652 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:44.581628    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:55:45.143304 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:45.143271    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l5nkc"]
Jul 10 15:55:45.143479 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:45.143415    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:45.143588 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:45.143557    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:45.146525 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:45.146496    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lh5cf"]
Jul 10 15:55:45.146658 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:45.146638    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:45.146783 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:45.146760    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:45.147210 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:45.147188    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xbfbt"]
Jul 10 15:55:45.147344 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:45.147328    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:45.147473 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:45.147452    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:45.569685 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:45.569654    2567 generic.go:356] "Generic (PLEG): container finished" podID="e64eb88a-9c8f-4ec8-858e-7c82904ab121" containerID="1ea61225391bab36ebb9c407becb4716f864e2662cce664987ebba93483fd0c1" exitCode=0
Jul 10 15:55:45.570059 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:45.569734    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwpkp" event={"ID":"e64eb88a-9c8f-4ec8-858e-7c82904ab121","Type":"ContainerDied","Data":"1ea61225391bab36ebb9c407becb4716f864e2662cce664987ebba93483fd0c1"}
Jul 10 15:55:46.399867 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:46.399675    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:46.400068 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:46.399697    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:46.400068 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:46.399953    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:46.400150 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:46.400087    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:46.984381 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:46.984341    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:46.984845 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:46.984513    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:46.984845 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:46.984588    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs podName:232217f5-52c2-4bd8-b0f7-73f950e6c2a5 nodeName:}" failed. No retries permitted until 2026-07-10 15:56:18.984569938 +0000 UTC m=+66.128103774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs") pod "network-metrics-daemon-lh5cf" (UID: "232217f5-52c2-4bd8-b0f7-73f950e6c2a5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 15:55:47.085124 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:47.085080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:47.085279 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:47.085223    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:47.085343 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:47.085280    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret podName:16d3e229-e008-438f-8f5a-13d9364e5b7b nodeName:}" failed. No retries permitted until 2026-07-10 15:56:19.085267202 +0000 UTC m=+66.228801019 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret") pod "global-pull-secret-syncer-l5nkc" (UID: "16d3e229-e008-438f-8f5a-13d9364e5b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 10 15:55:47.185882 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:47.185788    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmmp\" (UniqueName: \"kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp\") pod \"network-check-target-xbfbt\" (UID: \"25a66fdf-5eef-4fd3-b7b6-369254a11b3f\") " pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:47.186041 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:47.185980    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 15:55:47.186041 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:47.186005    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 15:55:47.186041 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:47.186022    2567 projected.go:196] Error preparing data for projected volume kube-api-access-qwmmp for pod openshift-network-diagnostics/network-check-target-xbfbt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:47.186184 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:47.186091    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp podName:25a66fdf-5eef-4fd3-b7b6-369254a11b3f nodeName:}" failed. No retries permitted until 2026-07-10 15:56:19.18607131 +0000 UTC m=+66.329605144 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwmmp" (UniqueName: "kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp") pod "network-check-target-xbfbt" (UID: "25a66fdf-5eef-4fd3-b7b6-369254a11b3f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 15:55:47.399470 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:47.399295    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:47.399470 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:47.399433    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l5nkc" podUID="16d3e229-e008-438f-8f5a-13d9364e5b7b"
Jul 10 15:55:48.399381 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:48.399341    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:48.399827 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:48.399341    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:48.399827 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:48.399492    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xbfbt" podUID="25a66fdf-5eef-4fd3-b7b6-369254a11b3f"
Jul 10 15:55:48.399933 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:48.399893    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lh5cf" podUID="232217f5-52c2-4bd8-b0f7-73f950e6c2a5"
Jul 10 15:55:49.134438 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.134391    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-156.ec2.internal" event="NodeReady"
Jul 10 15:55:49.134586 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.134568    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 15:55:49.241486 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.241364    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-z8vr5"]
Jul 10 15:55:49.244369 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.244345    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dch4q"]
Jul 10 15:55:49.244531 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.244506    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.246784 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.246762    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dch4q"
Jul 10 15:55:49.251624 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.251600    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 15:55:49.251734 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.251607    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zk7gp\""
Jul 10 15:55:49.252279 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.252252    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nh8x5\""
Jul 10 15:55:49.252542 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.252526    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 15:55:49.252634 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.252565    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 15:55:49.256735 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.256705    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 15:55:49.268303 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.268280    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z8vr5"]
Jul 10 15:55:49.271866 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.271839    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dch4q"]
Jul 10 15:55:49.274057 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.274034    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 15:55:49.399317 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.399229    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:55:49.401264 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.401239    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e21c179a-863d-43f0-9982-f10559073d45-cert\") pod \"ingress-canary-dch4q\" (UID: \"e21c179a-863d-43f0-9982-f10559073d45\") " pod="openshift-ingress-canary/ingress-canary-dch4q"
Jul 10 15:55:49.401623 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.401305    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxhvg\" (UniqueName: \"kubernetes.io/projected/e21c179a-863d-43f0-9982-f10559073d45-kube-api-access-gxhvg\") pod \"ingress-canary-dch4q\" (UID: \"e21c179a-863d-43f0-9982-f10559073d45\") " pod="openshift-ingress-canary/ingress-canary-dch4q"
Jul 10 15:55:49.401623 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.401385    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx4vg\" (UniqueName: \"kubernetes.io/projected/a6f5c915-40a8-4894-b83a-01985ca022fb-kube-api-access-qx4vg\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.401623 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.401489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.401623 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.401520    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6f5c915-40a8-4894-b83a-01985ca022fb-config-volume\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.401623 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.401546    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6f5c915-40a8-4894-b83a-01985ca022fb-tmp-dir\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.403954 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.403932    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 15:55:49.459360 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.459328    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-59786c5dc6-qhtdf"]
Jul 10 15:55:49.462083 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.462060    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.465299 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.465270    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 10 15:55:49.465458 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.465288    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 10 15:55:49.465458 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.465370    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 10 15:55:49.465578 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.465288    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 10 15:55:49.465644 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.465620    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 15:55:49.465769 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.465752    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 15:55:49.465869 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.465854    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pchcm\""
Jul 10 15:55:49.481800 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.481765    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59786c5dc6-qhtdf"]
Jul 10 15:55:49.502138 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.502107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.502138 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.502143    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6f5c915-40a8-4894-b83a-01985ca022fb-config-volume\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.502370 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.502170    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6f5c915-40a8-4894-b83a-01985ca022fb-tmp-dir\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.502370 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:49.502336    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 15:55:49.502502 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.502389    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e21c179a-863d-43f0-9982-f10559073d45-cert\") pod \"ingress-canary-dch4q\" (UID: \"e21c179a-863d-43f0-9982-f10559073d45\") " pod="openshift-ingress-canary/ingress-canary-dch4q"
Jul 10 15:55:49.502502 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:49.502445    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls podName:a6f5c915-40a8-4894-b83a-01985ca022fb nodeName:}" failed. No retries permitted until 2026-07-10 15:55:50.002394104 +0000 UTC m=+37.145927928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls") pod "dns-default-z8vr5" (UID: "a6f5c915-40a8-4894-b83a-01985ca022fb") : secret "dns-default-metrics-tls" not found
Jul 10 15:55:49.502619 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.502506    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxhvg\" (UniqueName: \"kubernetes.io/projected/e21c179a-863d-43f0-9982-f10559073d45-kube-api-access-gxhvg\") pod \"ingress-canary-dch4q\" (UID: \"e21c179a-863d-43f0-9982-f10559073d45\") " pod="openshift-ingress-canary/ingress-canary-dch4q"
Jul 10 15:55:49.502619 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.502513    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6f5c915-40a8-4894-b83a-01985ca022fb-tmp-dir\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.502619 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.502547    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx4vg\" (UniqueName: \"kubernetes.io/projected/a6f5c915-40a8-4894-b83a-01985ca022fb-kube-api-access-qx4vg\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.502843 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.502810    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6f5c915-40a8-4894-b83a-01985ca022fb-config-volume\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.507799 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.507765    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e21c179a-863d-43f0-9982-f10559073d45-cert\") pod \"ingress-canary-dch4q\" (UID: \"e21c179a-863d-43f0-9982-f10559073d45\") " pod="openshift-ingress-canary/ingress-canary-dch4q"
Jul 10 15:55:49.534717 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.534650    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx4vg\" (UniqueName: \"kubernetes.io/projected/a6f5c915-40a8-4894-b83a-01985ca022fb-kube-api-access-qx4vg\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:49.534901 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.534876    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxhvg\" (UniqueName: \"kubernetes.io/projected/e21c179a-863d-43f0-9982-f10559073d45-kube-api-access-gxhvg\") pod \"ingress-canary-dch4q\" (UID: \"e21c179a-863d-43f0-9982-f10559073d45\") " pod="openshift-ingress-canary/ingress-canary-dch4q"
Jul 10 15:55:49.564864 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.564806    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dch4q"
Jul 10 15:55:49.571159 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.571126    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-gphds"]
Jul 10 15:55:49.574721 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.574693    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"]
Jul 10 15:55:49.574839 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.574811    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.578189 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.578165    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9rb5"]
Jul 10 15:55:49.578588 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.578340    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"
Jul 10 15:55:49.578708 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.578690    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 10 15:55:49.579229 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.578987    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 10 15:55:49.579229 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.579112    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 15:55:49.579368 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.579286    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 15:55:49.579368 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.579288    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ctjcm\""
Jul 10 15:55:49.582579 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.581857    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 10 15:55:49.582579 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.581882    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 10 15:55:49.582579 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.582000    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 10 15:55:49.582579 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.582095    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9rb5"
Jul 10 15:55:49.582579 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.582194    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 10 15:55:49.582579 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.582379    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-h4h8x\""
Jul 10 15:55:49.586003 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.585939    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 10 15:55:49.586140 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.586039    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 10 15:55:49.586236 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.586221    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4nsv8\""
Jul 10 15:55:49.589998 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.589555    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"]
Jul 10 15:55:49.593603 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.593485    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 10 15:55:49.594000 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.593941    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-gphds"]
Jul 10 15:55:49.596746 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.596717    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9rb5"]
Jul 10 15:55:49.603293 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.603211    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.603293 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.603276    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-default-certificate\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.603542 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.603307    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzw8k\" (UniqueName: \"kubernetes.io/projected/ef2cf27d-6566-4821-9282-c222c9b35c50-kube-api-access-gzw8k\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.603542 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.603328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-stats-auth\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.603542 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.603342    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.663982 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.663136    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"]
Jul 10 15:55:49.668499 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.668467    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.677198 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.676803    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 15:55:49.679685 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.679659    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 15:55:49.682352 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.680731    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dtz22\""
Jul 10 15:55:49.682352 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.680912    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 15:55:49.684789 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.684727    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"]
Jul 10 15:55:49.685313 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.685275    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 15:55:49.704044 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704009    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4415dbe9-f215-4e32-b06a-f3fe0affea2c-tmp\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.704233 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.704233 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6dlz\" (UniqueName: \"kubernetes.io/projected/8ffb2acb-7443-471d-8755-3542c855072f-kube-api-access-w6dlz\") pod \"volume-data-source-validator-78495597f6-g9rb5\" (UID: \"8ffb2acb-7443-471d-8755-3542c855072f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9rb5"
Jul 10 15:55:49.704233 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704118    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4415dbe9-f215-4e32-b06a-f3fe0affea2c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.704233 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704146    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99sdp\" (UniqueName: \"kubernetes.io/projected/365169d4-d476-40f7-8ba9-3da44f46edde-kube-api-access-99sdp\") pod \"service-ca-operator-7695fc6c8c-xmk5d\" (UID: \"365169d4-d476-40f7-8ba9-3da44f46edde\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"
Jul 10 15:55:49.704233 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704171    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4415dbe9-f215-4e32-b06a-f3fe0affea2c-snapshots\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.704513 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:49.704233    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:50.204211299 +0000 UTC m=+37.347745138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : configmap references non-existent config key: service-ca.crt
Jul 10 15:55:49.704513 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704296    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-default-certificate\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.704513 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704342    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4415dbe9-f215-4e32-b06a-f3fe0affea2c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.704513 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4415dbe9-f215-4e32-b06a-f3fe0affea2c-serving-cert\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.704513 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704425    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzw8k\" (UniqueName: \"kubernetes.io/projected/ef2cf27d-6566-4821-9282-c222c9b35c50-kube-api-access-gzw8k\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.704513 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704451    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/365169d4-d476-40f7-8ba9-3da44f46edde-serving-cert\") pod \"service-ca-operator-7695fc6c8c-xmk5d\" (UID: \"365169d4-d476-40f7-8ba9-3da44f46edde\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"
Jul 10 15:55:49.704513 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704488    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-stats-auth\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.704513 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704512    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.704860 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704536    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wpvz\" (UniqueName: \"kubernetes.io/projected/4415dbe9-f215-4e32-b06a-f3fe0affea2c-kube-api-access-6wpvz\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.704860 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.704562    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/365169d4-d476-40f7-8ba9-3da44f46edde-config\") pod \"service-ca-operator-7695fc6c8c-xmk5d\" (UID: \"365169d4-d476-40f7-8ba9-3da44f46edde\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"
Jul 10 15:55:49.704860 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:49.704842    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 15:55:49.705003 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:49.704914    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:50.204896902 +0000 UTC m=+37.348430728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : secret "router-metrics-certs-default" not found
Jul 10 15:55:49.707689 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.707662    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-stats-auth\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.707807 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.707662    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-default-certificate\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.722273 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.722100    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzw8k\" (UniqueName: \"kubernetes.io/projected/ef2cf27d-6566-4821-9282-c222c9b35c50-kube-api-access-gzw8k\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:49.733171 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.733143    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dch4q"]
Jul 10 15:55:49.741687 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:49.741440    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode21c179a_863d_43f0_9982_f10559073d45.slice/crio-01d9a091dc1e53ae71f1fa8f1e65faa4e320707a341e60a8fffcf30690d3a6f7 WatchSource:0}: Error finding container 01d9a091dc1e53ae71f1fa8f1e65faa4e320707a341e60a8fffcf30690d3a6f7: Status 404 returned error can't find the container with id 01d9a091dc1e53ae71f1fa8f1e65faa4e320707a341e60a8fffcf30690d3a6f7
Jul 10 15:55:49.805730 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.805687    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcrh6\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-kube-api-access-wcrh6\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.805861 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.805778    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6dlz\" (UniqueName: \"kubernetes.io/projected/8ffb2acb-7443-471d-8755-3542c855072f-kube-api-access-w6dlz\") pod \"volume-data-source-validator-78495597f6-g9rb5\" (UID: \"8ffb2acb-7443-471d-8755-3542c855072f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9rb5"
Jul 10 15:55:49.805861 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.805832    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4415dbe9-f215-4e32-b06a-f3fe0affea2c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.805861 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.805858    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4415dbe9-f215-4e32-b06a-f3fe0affea2c-snapshots\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.806008 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.805903    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.806008 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.805940    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4415dbe9-f215-4e32-b06a-f3fe0affea2c-serving-cert\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.806008 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.805969    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-certificates\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.806124 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806007    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/365169d4-d476-40f7-8ba9-3da44f46edde-config\") pod \"service-ca-operator-7695fc6c8c-xmk5d\" (UID: \"365169d4-d476-40f7-8ba9-3da44f46edde\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"
Jul 10 15:55:49.806124 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806046    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-ca-trust-extracted\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.806124 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806089    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-bound-sa-token\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.806258 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806128    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-trusted-ca\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.806258 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806199    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4415dbe9-f215-4e32-b06a-f3fe0affea2c-tmp\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.806343 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806264    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99sdp\" (UniqueName: \"kubernetes.io/projected/365169d4-d476-40f7-8ba9-3da44f46edde-kube-api-access-99sdp\") pod \"service-ca-operator-7695fc6c8c-xmk5d\" (UID: \"365169d4-d476-40f7-8ba9-3da44f46edde\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"
Jul 10 15:55:49.806343 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806302    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/365169d4-d476-40f7-8ba9-3da44f46edde-serving-cert\") pod \"service-ca-operator-7695fc6c8c-xmk5d\" (UID: \"365169d4-d476-40f7-8ba9-3da44f46edde\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"
Jul 10 15:55:49.806343 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4415dbe9-f215-4e32-b06a-f3fe0affea2c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.806511 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806363    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-installation-pull-secrets\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.806511 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wpvz\" (UniqueName: \"kubernetes.io/projected/4415dbe9-f215-4e32-b06a-f3fe0affea2c-kube-api-access-6wpvz\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.806511 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806494    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-image-registry-private-configuration\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.806656 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806610    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4415dbe9-f215-4e32-b06a-f3fe0affea2c-snapshots\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.806710 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806653    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4415dbe9-f215-4e32-b06a-f3fe0affea2c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.806710 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.806665    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/365169d4-d476-40f7-8ba9-3da44f46edde-config\") pod \"service-ca-operator-7695fc6c8c-xmk5d\" (UID: \"365169d4-d476-40f7-8ba9-3da44f46edde\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"
Jul 10 15:55:49.807138 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.807108    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4415dbe9-f215-4e32-b06a-f3fe0affea2c-tmp\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.807455 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.807433    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4415dbe9-f215-4e32-b06a-f3fe0affea2c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.809334 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.809317    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/365169d4-d476-40f7-8ba9-3da44f46edde-serving-cert\") pod \"service-ca-operator-7695fc6c8c-xmk5d\" (UID: \"365169d4-d476-40f7-8ba9-3da44f46edde\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"
Jul 10 15:55:49.809387 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.809360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4415dbe9-f215-4e32-b06a-f3fe0affea2c-serving-cert\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.820207 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.820171    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6dlz\" (UniqueName: \"kubernetes.io/projected/8ffb2acb-7443-471d-8755-3542c855072f-kube-api-access-w6dlz\") pod \"volume-data-source-validator-78495597f6-g9rb5\" (UID: \"8ffb2acb-7443-471d-8755-3542c855072f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9rb5"
Jul 10 15:55:49.820337 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.820211    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99sdp\" (UniqueName: \"kubernetes.io/projected/365169d4-d476-40f7-8ba9-3da44f46edde-kube-api-access-99sdp\") pod \"service-ca-operator-7695fc6c8c-xmk5d\" (UID: \"365169d4-d476-40f7-8ba9-3da44f46edde\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"
Jul 10 15:55:49.821998 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.821976    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wpvz\" (UniqueName: \"kubernetes.io/projected/4415dbe9-f215-4e32-b06a-f3fe0affea2c-kube-api-access-6wpvz\") pod \"insights-operator-68d9dd94cf-gphds\" (UID: \"4415dbe9-f215-4e32-b06a-f3fe0affea2c\") " pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.896199 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.896163    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-gphds"
Jul 10 15:55:49.904398 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.904305    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"
Jul 10 15:55:49.907628 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.907603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-installation-pull-secrets\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.907712 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.907658    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-image-registry-private-configuration\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.907712 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.907697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcrh6\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-kube-api-access-wcrh6\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.907785 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.907723    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.907785 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.907761    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-certificates\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.907855 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.907801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-ca-trust-extracted\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.907855 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.907826    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-bound-sa-token\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.907855 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:49.907832    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 15:55:49.907855 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.907849    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-trusted-ca\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.907855 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:49.907854    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68ff9d4cf8-f9b9h: secret "image-registry-tls" not found
Jul 10 15:55:49.908082 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:49.907915    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls podName:949e1ae5-ccf7-479c-a0f1-526bd9a254bf nodeName:}" failed. No retries permitted until 2026-07-10 15:55:50.407896016 +0000 UTC m=+37.551429854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls") pod "image-registry-68ff9d4cf8-f9b9h" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf") : secret "image-registry-tls" not found
Jul 10 15:55:49.908231 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.908211    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-ca-trust-extracted\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.908871 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.908853    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-trusted-ca\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.908969 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.908890    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-certificates\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.909949 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.909928    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9rb5"
Jul 10 15:55:49.910173 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.910155    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-installation-pull-secrets\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.910217 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.910192    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-image-registry-private-configuration\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.918716 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.918685    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcrh6\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-kube-api-access-wcrh6\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:49.919323 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:49.919301    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-bound-sa-token\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:50.008545 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.008498    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:50.008707 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:50.008688    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 15:55:50.008822 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:50.008760    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls podName:a6f5c915-40a8-4894-b83a-01985ca022fb nodeName:}" failed. No retries permitted until 2026-07-10 15:55:51.008739706 +0000 UTC m=+38.152273540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls") pod "dns-default-z8vr5" (UID: "a6f5c915-40a8-4894-b83a-01985ca022fb") : secret "dns-default-metrics-tls" not found
Jul 10 15:55:50.068283 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.068252    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-gphds"]
Jul 10 15:55:50.073316 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:50.073277    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4415dbe9_f215_4e32_b06a_f3fe0affea2c.slice/crio-3b4507e700a54022aef6da73c3fcb09f8ccc0b9d471248603b956a839d36707a WatchSource:0}: Error finding container 3b4507e700a54022aef6da73c3fcb09f8ccc0b9d471248603b956a839d36707a: Status 404 returned error can't find the container with id 3b4507e700a54022aef6da73c3fcb09f8ccc0b9d471248603b956a839d36707a
Jul 10 15:55:50.103709 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.103674    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d"]
Jul 10 15:55:50.107137 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:50.107110    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod365169d4_d476_40f7_8ba9_3da44f46edde.slice/crio-b497fbc201ad252bea7380e8a516d989afbfc3ef9ce64f3a5ba881188571dc83 WatchSource:0}: Error finding container b497fbc201ad252bea7380e8a516d989afbfc3ef9ce64f3a5ba881188571dc83: Status 404 returned error can't find the container with id b497fbc201ad252bea7380e8a516d989afbfc3ef9ce64f3a5ba881188571dc83
Jul 10 15:55:50.133074 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.133045    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9rb5"]
Jul 10 15:55:50.136551 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:55:50.136524    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ffb2acb_7443_471d_8755_3542c855072f.slice/crio-455e6f1a04d50d84fec4d0876240eda32217cdb269564d5c93aadfacd8f29d0d WatchSource:0}: Error finding container 455e6f1a04d50d84fec4d0876240eda32217cdb269564d5c93aadfacd8f29d0d: Status 404 returned error can't find the container with id 455e6f1a04d50d84fec4d0876240eda32217cdb269564d5c93aadfacd8f29d0d
Jul 10 15:55:50.209644 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.209618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:50.209828 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.209669    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:50.209828 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:50.209802    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:51.209785 +0000 UTC m=+38.353318822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : configmap references non-existent config key: service-ca.crt
Jul 10 15:55:50.209828 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:50.209804    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 15:55:50.209938 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:50.209840    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:51.2098336 +0000 UTC m=+38.353367424 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : secret "router-metrics-certs-default" not found
Jul 10 15:55:50.400002 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.399958    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:55:50.400194 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.400162    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:55:50.409018 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.408322    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 15:55:50.409018 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.408731    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 15:55:50.409018 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.408744    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8rkdc\""
Jul 10 15:55:50.409018 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.408767    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2tf4w\""
Jul 10 15:55:50.409018 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.408803    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 15:55:50.411815 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.411795    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:50.411936 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:50.411919    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 15:55:50.411936 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:50.411935    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68ff9d4cf8-f9b9h: secret "image-registry-tls" not found
Jul 10 15:55:50.412043 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:50.411986    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls podName:949e1ae5-ccf7-479c-a0f1-526bd9a254bf nodeName:}" failed. No retries permitted until 2026-07-10 15:55:51.41197102 +0000 UTC m=+38.555504850 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls") pod "image-registry-68ff9d4cf8-f9b9h" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf") : secret "image-registry-tls" not found
Jul 10 15:55:50.584514 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.584399    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-gphds" event={"ID":"4415dbe9-f215-4e32-b06a-f3fe0affea2c","Type":"ContainerStarted","Data":"3b4507e700a54022aef6da73c3fcb09f8ccc0b9d471248603b956a839d36707a"}
Jul 10 15:55:50.585754 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.585722    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d" event={"ID":"365169d4-d476-40f7-8ba9-3da44f46edde","Type":"ContainerStarted","Data":"b497fbc201ad252bea7380e8a516d989afbfc3ef9ce64f3a5ba881188571dc83"}
Jul 10 15:55:50.588951 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.588923    2567 generic.go:356] "Generic (PLEG): container finished" podID="e64eb88a-9c8f-4ec8-858e-7c82904ab121" containerID="c9904571b85083d4c1ab9a6be52f26e95b2dce1f6e5f79424be3f35b427bf962" exitCode=0
Jul 10 15:55:50.589185 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.589160    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwpkp" event={"ID":"e64eb88a-9c8f-4ec8-858e-7c82904ab121","Type":"ContainerDied","Data":"c9904571b85083d4c1ab9a6be52f26e95b2dce1f6e5f79424be3f35b427bf962"}
Jul 10 15:55:50.591237 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.591213    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9rb5" event={"ID":"8ffb2acb-7443-471d-8755-3542c855072f","Type":"ContainerStarted","Data":"455e6f1a04d50d84fec4d0876240eda32217cdb269564d5c93aadfacd8f29d0d"}
Jul 10 15:55:50.594707 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:50.594685    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dch4q" event={"ID":"e21c179a-863d-43f0-9982-f10559073d45","Type":"ContainerStarted","Data":"01d9a091dc1e53ae71f1fa8f1e65faa4e320707a341e60a8fffcf30690d3a6f7"}
Jul 10 15:55:51.017619 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:51.017575    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:51.017880 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:51.017863    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 15:55:51.017960 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:51.017931    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls podName:a6f5c915-40a8-4894-b83a-01985ca022fb nodeName:}" failed. No retries permitted until 2026-07-10 15:55:53.017911199 +0000 UTC m=+40.161445028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls") pod "dns-default-z8vr5" (UID: "a6f5c915-40a8-4894-b83a-01985ca022fb") : secret "dns-default-metrics-tls" not found
Jul 10 15:55:51.219358 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:51.219306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:51.219547 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:51.219389    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:51.220169 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:51.219616    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 15:55:51.220169 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:51.219687    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:53.219668732 +0000 UTC m=+40.363202555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : secret "router-metrics-certs-default" not found
Jul 10 15:55:51.220169 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:51.219773    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:53.219762398 +0000 UTC m=+40.363296224 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : configmap references non-existent config key: service-ca.crt
Jul 10 15:55:51.421731 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:51.421581    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:51.422296 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:51.421737    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 15:55:51.422296 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:51.421761    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68ff9d4cf8-f9b9h: secret "image-registry-tls" not found
Jul 10 15:55:51.422296 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:51.421830    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls podName:949e1ae5-ccf7-479c-a0f1-526bd9a254bf nodeName:}" failed. No retries permitted until 2026-07-10 15:55:53.421810584 +0000 UTC m=+40.565344425 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls") pod "image-registry-68ff9d4cf8-f9b9h" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf") : secret "image-registry-tls" not found
Jul 10 15:55:51.604011 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:51.603972    2567 generic.go:356] "Generic (PLEG): container finished" podID="e64eb88a-9c8f-4ec8-858e-7c82904ab121" containerID="532a295e4fb25cc6fb9bf2197934cc326fd6a9cb50ff366af11a949c8281839c" exitCode=0
Jul 10 15:55:51.604181 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:51.604030    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwpkp" event={"ID":"e64eb88a-9c8f-4ec8-858e-7c82904ab121","Type":"ContainerDied","Data":"532a295e4fb25cc6fb9bf2197934cc326fd6a9cb50ff366af11a949c8281839c"}
Jul 10 15:55:53.037223 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:53.037169    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:53.037755 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:53.037320    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 15:55:53.037755 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:53.037375    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls podName:a6f5c915-40a8-4894-b83a-01985ca022fb nodeName:}" failed. No retries permitted until 2026-07-10 15:55:57.03736063 +0000 UTC m=+44.180894446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls") pod "dns-default-z8vr5" (UID: "a6f5c915-40a8-4894-b83a-01985ca022fb") : secret "dns-default-metrics-tls" not found
Jul 10 15:55:53.239079 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:53.239035    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:53.239079 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:53.239089    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:53.239309 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:53.239191    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 15:55:53.239309 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:53.239216    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:57.239197586 +0000 UTC m=+44.382731403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : configmap references non-existent config key: service-ca.crt
Jul 10 15:55:53.239309 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:53.239301    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:57.239293505 +0000 UTC m=+44.382827323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : secret "router-metrics-certs-default" not found
Jul 10 15:55:53.441064 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:53.440809    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:53.441223 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:53.440983    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 15:55:53.441223 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:53.441097    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68ff9d4cf8-f9b9h: secret "image-registry-tls" not found
Jul 10 15:55:53.441223 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:53.441169    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls podName:949e1ae5-ccf7-479c-a0f1-526bd9a254bf nodeName:}" failed. No retries permitted until 2026-07-10 15:55:57.441150586 +0000 UTC m=+44.584684408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls") pod "image-registry-68ff9d4cf8-f9b9h" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf") : secret "image-registry-tls" not found
Jul 10 15:55:54.614844 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:54.614760    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-gphds" event={"ID":"4415dbe9-f215-4e32-b06a-f3fe0affea2c","Type":"ContainerStarted","Data":"bd73e462bd63a1d205bbe036e56936a15cf6eb8b8bc498bbc296b2c8081df2a2"}
Jul 10 15:55:54.617335 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:54.617303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d" event={"ID":"365169d4-d476-40f7-8ba9-3da44f46edde","Type":"ContainerStarted","Data":"407db4628629e577d4295aa477d1fc8a52ef850a403622bcdc741abb17a19fa2"}
Jul 10 15:55:54.622584 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:54.621954    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lwpkp" event={"ID":"e64eb88a-9c8f-4ec8-858e-7c82904ab121","Type":"ContainerStarted","Data":"57cd77cc5a01b65e76dc6372db0148e35943cde8b541414eed91e756c0a6b364"}
Jul 10 15:55:54.625071 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:54.625030    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9rb5" event={"ID":"8ffb2acb-7443-471d-8755-3542c855072f","Type":"ContainerStarted","Data":"28a9316a4f547adfa889dc69490f7f15be5fec4b2987dae0a6cd666a51579460"}
Jul 10 15:55:54.626468 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:54.626446    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dch4q" event={"ID":"e21c179a-863d-43f0-9982-f10559073d45","Type":"ContainerStarted","Data":"da8444e0d57e5ce09ad7e1ae23893f83f0b459d84f2da4253a850db118aa174d"}
Jul 10 15:55:54.657675 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:54.657615    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-gphds" podStartSLOduration=1.344085129 podStartE2EDuration="5.657592086s" totalImagesPullingTime="4.313506957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:54.634388598 +0000 UTC m=+41.777922439" watchObservedRunningTime="2026-07-10 15:55:54.657592086 +0000 UTC m=+41.801125927"
Jul 10 15:55:54.658002 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:54.657970    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lwpkp" podStartSLOduration=12.055324667 podStartE2EDuration="41.657958338s" totalImagesPullingTime="29.602633671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:54.656785511 +0000 UTC m=+41.800319354" watchObservedRunningTime="2026-07-10 15:55:54.657958338 +0000 UTC m=+41.801492177"
Jul 10 15:55:54.674901 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:54.674843    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dch4q" podStartSLOduration=1.036177029 podStartE2EDuration="5.674824027s" totalImagesPullingTime="4.638646998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:54.673495434 +0000 UTC m=+41.817029275" watchObservedRunningTime="2026-07-10 15:55:54.674824027 +0000 UTC m=+41.818357868"
Jul 10 15:55:54.691355 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:54.691216    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d" podStartSLOduration=1.416041037 podStartE2EDuration="5.691194399s" totalImagesPullingTime="4.275153362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:54.690759301 +0000 UTC m=+41.834293142" watchObservedRunningTime="2026-07-10 15:55:54.691194399 +0000 UTC m=+41.834728241"
Jul 10 15:55:54.707924 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:54.707862    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9rb5" podStartSLOduration=1.461984103 podStartE2EDuration="5.707841364s" totalImagesPullingTime="4.245857261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:54.706862806 +0000 UTC m=+41.850396646" watchObservedRunningTime="2026-07-10 15:55:54.707841364 +0000 UTC m=+41.851375205"
Jul 10 15:55:56.161877 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.161848    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sjb7f"]
Jul 10 15:55:56.182878 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.182844    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sjb7f"]
Jul 10 15:55:56.183036 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.182966    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.185515 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.185490    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 15:55:56.185663 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.185523    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6kc2\""
Jul 10 15:55:56.185663 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.185534    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 15:55:56.267642 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.267599    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-data-volume\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.267806 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.267651    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dghbg\" (UniqueName: \"kubernetes.io/projected/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-api-access-dghbg\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.267806 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.267771    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.267806 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.267796    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-crio-socket\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.267911 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.267833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.368425 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.368374    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.368576 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.368454    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-data-volume\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.368576 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.368479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dghbg\" (UniqueName: \"kubernetes.io/projected/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-api-access-dghbg\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.368576 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:56.368503    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 15:55:56.368576 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.368546    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.368576 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:56.368570    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm podName:194db3c8-57e0-4bec-bef7-c2ef3ca7fce7 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:56.868554494 +0000 UTC m=+44.012088311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sjb7f" (UID: "194db3c8-57e0-4bec-bef7-c2ef3ca7fce7") : configmap "kube-rbac-proxy" not found
Jul 10 15:55:56.368797 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.368601    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-crio-socket\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.368797 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:56.368620    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 15:55:56.368797 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:56.368665    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls podName:194db3c8-57e0-4bec-bef7-c2ef3ca7fce7 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:56.868649351 +0000 UTC m=+44.012183175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sjb7f" (UID: "194db3c8-57e0-4bec-bef7-c2ef3ca7fce7") : secret "insights-runtime-extractor-tls" not found
Jul 10 15:55:56.368927 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.368806    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-crio-socket\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.368927 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.368920    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-data-volume\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.383474 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.383434    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dghbg\" (UniqueName: \"kubernetes.io/projected/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-api-access-dghbg\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.871474 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.871434    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.871625 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:56.871550    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:56.871625 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:56.871561    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 15:55:56.871625 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:56.871624    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm podName:194db3c8-57e0-4bec-bef7-c2ef3ca7fce7 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:57.871608635 +0000 UTC m=+45.015142456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sjb7f" (UID: "194db3c8-57e0-4bec-bef7-c2ef3ca7fce7") : configmap "kube-rbac-proxy" not found
Jul 10 15:55:56.871750 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:56.871636    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 15:55:56.871750 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:56.871675    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls podName:194db3c8-57e0-4bec-bef7-c2ef3ca7fce7 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:57.871664522 +0000 UTC m=+45.015198343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sjb7f" (UID: "194db3c8-57e0-4bec-bef7-c2ef3ca7fce7") : secret "insights-runtime-extractor-tls" not found
Jul 10 15:55:57.073103 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:57.073062    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:55:57.073329 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.073239    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 15:55:57.073329 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.073321    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls podName:a6f5c915-40a8-4894-b83a-01985ca022fb nodeName:}" failed. No retries permitted until 2026-07-10 15:56:05.07329942 +0000 UTC m=+52.216833249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls") pod "dns-default-z8vr5" (UID: "a6f5c915-40a8-4894-b83a-01985ca022fb") : secret "dns-default-metrics-tls" not found
Jul 10 15:55:57.274172 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:57.274135    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:57.274667 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:57.274277    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:55:57.274667 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.274317    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 15:55:57.274667 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.274394    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:56:05.274373799 +0000 UTC m=+52.417907630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : secret "router-metrics-certs-default" not found
Jul 10 15:55:57.274667 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.274445    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:56:05.274428323 +0000 UTC m=+52.417962154 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : configmap references non-existent config key: service-ca.crt
Jul 10 15:55:57.476322 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:57.476283    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:55:57.476542 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.476445    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 15:55:57.476542 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.476461    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68ff9d4cf8-f9b9h: secret "image-registry-tls" not found
Jul 10 15:55:57.476542 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.476522    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls podName:949e1ae5-ccf7-479c-a0f1-526bd9a254bf nodeName:}" failed. No retries permitted until 2026-07-10 15:56:05.476504277 +0000 UTC m=+52.620038093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls") pod "image-registry-68ff9d4cf8-f9b9h" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf") : secret "image-registry-tls" not found
Jul 10 15:55:57.879538 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:57.879500    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:57.879749 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.879633    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 15:55:57.879749 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:57.879654    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:57.879749 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.879703    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm podName:194db3c8-57e0-4bec-bef7-c2ef3ca7fce7 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:59.87968292 +0000 UTC m=+47.023216737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sjb7f" (UID: "194db3c8-57e0-4bec-bef7-c2ef3ca7fce7") : configmap "kube-rbac-proxy" not found
Jul 10 15:55:57.879920 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.879761    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 15:55:57.879920 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:57.879797    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls podName:194db3c8-57e0-4bec-bef7-c2ef3ca7fce7 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:59.879789404 +0000 UTC m=+47.023323221 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sjb7f" (UID: "194db3c8-57e0-4bec-bef7-c2ef3ca7fce7") : secret "insights-runtime-extractor-tls" not found
Jul 10 15:55:59.091364 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:59.091337    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-htc8d_76d52889-8bfd-412c-b4fe-3943f4e074a0/dns-node-resolver/0.log"
Jul 10 15:55:59.890064 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:59.890035    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kdsx4_9fc05a30-9841-4627-a43e-5591964cce62/node-ca/0.log"
Jul 10 15:55:59.896382 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:59.896354    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:59.896500 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:59.896484    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 15:55:59.896500 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:55:59.896494    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:55:59.896566 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:59.896543    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm podName:194db3c8-57e0-4bec-bef7-c2ef3ca7fce7 nodeName:}" failed. No retries permitted until 2026-07-10 15:56:03.896526042 +0000 UTC m=+51.040059858 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sjb7f" (UID: "194db3c8-57e0-4bec-bef7-c2ef3ca7fce7") : configmap "kube-rbac-proxy" not found
Jul 10 15:55:59.896617 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:59.896597    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 15:55:59.896659 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:55:59.896642    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls podName:194db3c8-57e0-4bec-bef7-c2ef3ca7fce7 nodeName:}" failed. No retries permitted until 2026-07-10 15:56:03.896630852 +0000 UTC m=+51.040164673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sjb7f" (UID: "194db3c8-57e0-4bec-bef7-c2ef3ca7fce7") : secret "insights-runtime-extractor-tls" not found
Jul 10 15:56:00.695129 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:00.695099    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dch4q_e21c179a-863d-43f0-9982-f10559073d45/serve-healthcheck-canary/0.log"
Jul 10 15:56:03.929646 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:03.929597    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:56:03.930045 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:03.929716    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:56:03.930045 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:03.929735    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 15:56:03.930045 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:03.929804    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm podName:194db3c8-57e0-4bec-bef7-c2ef3ca7fce7 nodeName:}" failed. No retries permitted until 2026-07-10 15:56:11.929788709 +0000 UTC m=+59.073322529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sjb7f" (UID: "194db3c8-57e0-4bec-bef7-c2ef3ca7fce7") : configmap "kube-rbac-proxy" not found
Jul 10 15:56:03.930045 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:03.929832    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 15:56:03.930045 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:03.929871    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls podName:194db3c8-57e0-4bec-bef7-c2ef3ca7fce7 nodeName:}" failed. No retries permitted until 2026-07-10 15:56:11.929859821 +0000 UTC m=+59.073393639 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sjb7f" (UID: "194db3c8-57e0-4bec-bef7-c2ef3ca7fce7") : secret "insights-runtime-extractor-tls" not found
Jul 10 15:56:05.139534 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:05.139496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:56:05.139940 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:05.139657    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 15:56:05.139940 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:05.139725    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls podName:a6f5c915-40a8-4894-b83a-01985ca022fb nodeName:}" failed. No retries permitted until 2026-07-10 15:56:21.139707761 +0000 UTC m=+68.283241581 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls") pod "dns-default-z8vr5" (UID: "a6f5c915-40a8-4894-b83a-01985ca022fb") : secret "dns-default-metrics-tls" not found
Jul 10 15:56:05.341537 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:05.341484    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:56:05.341746 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:05.341585    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:56:05.341746 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:05.341674    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:56:21.341655192 +0000 UTC m=+68.485189016 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : configmap references non-existent config key: service-ca.crt
Jul 10 15:56:05.341746 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:05.341730    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 10 15:56:05.341883 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:05.341789    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs podName:ef2cf27d-6566-4821-9282-c222c9b35c50 nodeName:}" failed. No retries permitted until 2026-07-10 15:56:21.341776662 +0000 UTC m=+68.485310483 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs") pod "router-default-59786c5dc6-qhtdf" (UID: "ef2cf27d-6566-4821-9282-c222c9b35c50") : secret "router-metrics-certs-default" not found
Jul 10 15:56:05.543808 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:05.543768    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:56:05.544136 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:05.543919    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 15:56:05.544136 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:05.543936    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68ff9d4cf8-f9b9h: secret "image-registry-tls" not found
Jul 10 15:56:05.544136 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:05.544001    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls podName:949e1ae5-ccf7-479c-a0f1-526bd9a254bf nodeName:}" failed. No retries permitted until 2026-07-10 15:56:21.543986854 +0000 UTC m=+68.687520675 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls") pod "image-registry-68ff9d4cf8-f9b9h" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf") : secret "image-registry-tls" not found
Jul 10 15:56:11.998713 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:11.998666    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:56:11.999208 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:11.998743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:56:11.999208 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:11.998835    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 15:56:11.999208 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:11.998926    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls podName:194db3c8-57e0-4bec-bef7-c2ef3ca7fce7 nodeName:}" failed. No retries permitted until 2026-07-10 15:56:27.998905891 +0000 UTC m=+75.142439715 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sjb7f" (UID: "194db3c8-57e0-4bec-bef7-c2ef3ca7fce7") : secret "insights-runtime-extractor-tls" not found
Jul 10 15:56:11.999360 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:11.999341    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:56:16.585268 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:16.585235    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qz2jw"
Jul 10 15:56:19.061165 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.061121    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:56:19.064274 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.064251    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 15:56:19.074204 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.074169    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232217f5-52c2-4bd8-b0f7-73f950e6c2a5-metrics-certs\") pod \"network-metrics-daemon-lh5cf\" (UID: \"232217f5-52c2-4bd8-b0f7-73f950e6c2a5\") " pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:56:19.161693 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.161649    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:56:19.164771 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.164750    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 15:56:19.174712 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.174673    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d3e229-e008-438f-8f5a-13d9364e5b7b-original-pull-secret\") pod \"global-pull-secret-syncer-l5nkc\" (UID: \"16d3e229-e008-438f-8f5a-13d9364e5b7b\") " pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:56:19.216255 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.216225    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8rkdc\""
Jul 10 15:56:19.223929 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.223900    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lh5cf"
Jul 10 15:56:19.263056 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.263024    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmmp\" (UniqueName: \"kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp\") pod \"network-check-target-xbfbt\" (UID: \"25a66fdf-5eef-4fd3-b7b6-369254a11b3f\") " pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:56:19.266364 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.266262    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 15:56:19.278914 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.278883    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 15:56:19.286968 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.286933    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwmmp\" (UniqueName: \"kubernetes.io/projected/25a66fdf-5eef-4fd3-b7b6-369254a11b3f-kube-api-access-qwmmp\") pod \"network-check-target-xbfbt\" (UID: \"25a66fdf-5eef-4fd3-b7b6-369254a11b3f\") " pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:56:19.363454 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.363354    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lh5cf"]
Jul 10 15:56:19.366845 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:56:19.366818    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod232217f5_52c2_4bd8_b0f7_73f950e6c2a5.slice/crio-8aa10003a13669edc7a7eb8c5d3db6756af3c1675a414c023dcdd44fb14cffed WatchSource:0}: Error finding container 8aa10003a13669edc7a7eb8c5d3db6756af3c1675a414c023dcdd44fb14cffed: Status 404 returned error can't find the container with id 8aa10003a13669edc7a7eb8c5d3db6756af3c1675a414c023dcdd44fb14cffed
Jul 10 15:56:19.409652 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.409615    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l5nkc"
Jul 10 15:56:19.522389 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.522357    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2tf4w\""
Jul 10 15:56:19.530044 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.530013    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:56:19.563625 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.563587    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l5nkc"]
Jul 10 15:56:19.567867 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:56:19.567832    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d3e229_e008_438f_8f5a_13d9364e5b7b.slice/crio-dc72adad376f7be2b30ce879d9141e414fd59c7d38d5cfc174733ed88d84097b WatchSource:0}: Error finding container dc72adad376f7be2b30ce879d9141e414fd59c7d38d5cfc174733ed88d84097b: Status 404 returned error can't find the container with id dc72adad376f7be2b30ce879d9141e414fd59c7d38d5cfc174733ed88d84097b
Jul 10 15:56:19.687765 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.687733    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xbfbt"]
Jul 10 15:56:19.690330 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.690300    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lh5cf" event={"ID":"232217f5-52c2-4bd8-b0f7-73f950e6c2a5","Type":"ContainerStarted","Data":"8aa10003a13669edc7a7eb8c5d3db6756af3c1675a414c023dcdd44fb14cffed"}
Jul 10 15:56:19.690832 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:56:19.690805    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a66fdf_5eef_4fd3_b7b6_369254a11b3f.slice/crio-525a2178594ddb20ebd23648473c938d4359741ba672c3201e7b3a24b73235ce WatchSource:0}: Error finding container 525a2178594ddb20ebd23648473c938d4359741ba672c3201e7b3a24b73235ce: Status 404 returned error can't find the container with id 525a2178594ddb20ebd23648473c938d4359741ba672c3201e7b3a24b73235ce
Jul 10 15:56:19.693541 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:19.693512    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l5nkc" event={"ID":"16d3e229-e008-438f-8f5a-13d9364e5b7b","Type":"ContainerStarted","Data":"dc72adad376f7be2b30ce879d9141e414fd59c7d38d5cfc174733ed88d84097b"}
Jul 10 15:56:20.700600 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:20.700522    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xbfbt" event={"ID":"25a66fdf-5eef-4fd3-b7b6-369254a11b3f","Type":"ContainerStarted","Data":"525a2178594ddb20ebd23648473c938d4359741ba672c3201e7b3a24b73235ce"}
Jul 10 15:56:21.181782 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.181678    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:56:21.185021 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.184955    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6f5c915-40a8-4894-b83a-01985ca022fb-metrics-tls\") pod \"dns-default-z8vr5\" (UID: \"a6f5c915-40a8-4894-b83a-01985ca022fb\") " pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:56:21.362771 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.362566    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nh8x5\""
Jul 10 15:56:21.368292 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.368264    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:56:21.383624 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.383584    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:56:21.383766 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.383654    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:56:21.384935 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.384905    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2cf27d-6566-4821-9282-c222c9b35c50-service-ca-bundle\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:56:21.391242 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.391215    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef2cf27d-6566-4821-9282-c222c9b35c50-metrics-certs\") pod \"router-default-59786c5dc6-qhtdf\" (UID: \"ef2cf27d-6566-4821-9282-c222c9b35c50\") " pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:56:21.582917 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.582694    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pchcm\""
Jul 10 15:56:21.584080 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.584000    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:56:21.584080 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.584016    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z8vr5"]
Jul 10 15:56:21.585295 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.585202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:56:21.586541 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:56:21.586512    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6f5c915_40a8_4894_b83a_01985ca022fb.slice/crio-fcd11d9dc9b8d2045ad0f9e0fb44674dc2dea73ac5d3be002de5d8dd862e65b2 WatchSource:0}: Error finding container fcd11d9dc9b8d2045ad0f9e0fb44674dc2dea73ac5d3be002de5d8dd862e65b2: Status 404 returned error can't find the container with id fcd11d9dc9b8d2045ad0f9e0fb44674dc2dea73ac5d3be002de5d8dd862e65b2
Jul 10 15:56:21.588581 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.588489    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls\") pod \"image-registry-68ff9d4cf8-f9b9h\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") " pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:56:21.710089 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.709737    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lh5cf" event={"ID":"232217f5-52c2-4bd8-b0f7-73f950e6c2a5","Type":"ContainerStarted","Data":"ebd3f36e32080d368fbf049c507e4dd8463f7c0d6a442385339903ac0a30f711"}
Jul 10 15:56:21.710089 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.709796    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lh5cf" event={"ID":"232217f5-52c2-4bd8-b0f7-73f950e6c2a5","Type":"ContainerStarted","Data":"d75bcee230a28ce92fee046119117bf875816ebe8ea0130c1796286abe6f358b"}
Jul 10 15:56:21.711927 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.711887    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z8vr5" event={"ID":"a6f5c915-40a8-4894-b83a-01985ca022fb","Type":"ContainerStarted","Data":"fcd11d9dc9b8d2045ad0f9e0fb44674dc2dea73ac5d3be002de5d8dd862e65b2"}
Jul 10 15:56:21.745796 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.745736    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lh5cf" podStartSLOduration=66.877867087 podStartE2EDuration="1m8.745714855s" totalImagesPullingTime="1.867847768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:21.729288316 +0000 UTC m=+68.872822155" watchObservedRunningTime="2026-07-10 15:56:21.745714855 +0000 UTC m=+68.889248694"
Jul 10 15:56:21.746051 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.746032    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59786c5dc6-qhtdf"]
Jul 10 15:56:21.752217 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:56:21.752174    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef2cf27d_6566_4821_9282_c222c9b35c50.slice/crio-86f7ad75ab5b912f1f07011f3d8989d4e8f9bac436f26fa2b4cb1f5ee758bb53 WatchSource:0}: Error finding container 86f7ad75ab5b912f1f07011f3d8989d4e8f9bac436f26fa2b4cb1f5ee758bb53: Status 404 returned error can't find the container with id 86f7ad75ab5b912f1f07011f3d8989d4e8f9bac436f26fa2b4cb1f5ee758bb53
Jul 10 15:56:21.789318 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.789287    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dtz22\""
Jul 10 15:56:21.798695 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.798662    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:56:21.999696 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:21.999530    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"]
Jul 10 15:56:22.006649 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:56:22.006613    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949e1ae5_ccf7_479c_a0f1_526bd9a254bf.slice/crio-1fa4636a1d49f2c84d5426d4cd5cfca135035c84e09d67c161ef599510cfd783 WatchSource:0}: Error finding container 1fa4636a1d49f2c84d5426d4cd5cfca135035c84e09d67c161ef599510cfd783: Status 404 returned error can't find the container with id 1fa4636a1d49f2c84d5426d4cd5cfca135035c84e09d67c161ef599510cfd783
Jul 10 15:56:22.279549 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.279392    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"]
Jul 10 15:56:22.307427 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.307365    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-bdtkv"]
Jul 10 15:56:22.311255 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.311175    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-bdtkv"
Jul 10 15:56:22.326135 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.326108    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 10 15:56:22.327058 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.327040    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 10 15:56:22.332754 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.332724    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-kj2sz\""
Jul 10 15:56:22.353111 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.353076    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w"]
Jul 10 15:56:22.355695 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.355653    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-bdtkv"]
Jul 10 15:56:22.355870 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.355790    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w"
Jul 10 15:56:22.359780 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:22.359739    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"managed-serviceaccount-dockercfg-52w6s\" is forbidden: User \"system:node:ip-10-0-135-156.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-135-156.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-52w6s\"" type="*v1.Secret"
Jul 10 15:56:22.359984 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:22.359824    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"open-cluster-management-image-pull-credentials\" is forbidden: User \"system:node:ip-10-0-135-156.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-135-156.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret"
Jul 10 15:56:22.363111 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.363079    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 10 15:56:22.363375 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.363357    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 10 15:56:22.364360 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.364337    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 10 15:56:22.393697 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.393653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpl2q\" (UniqueName: \"kubernetes.io/projected/97be136b-397f-4d36-a5b5-5697338ec9f2-kube-api-access-gpl2q\") pod \"managed-serviceaccount-addon-agent-5c7b67f46d-qh46w\" (UID: \"97be136b-397f-4d36-a5b5-5697338ec9f2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w"
Jul 10 15:56:22.393883 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.393716    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ebf7138d-5fef-4905-9e1f-64c011efc16d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bdtkv\" (UID: \"ebf7138d-5fef-4905-9e1f-64c011efc16d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bdtkv"
Jul 10 15:56:22.393883 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.393752    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/97be136b-397f-4d36-a5b5-5697338ec9f2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c7b67f46d-qh46w\" (UID: \"97be136b-397f-4d36-a5b5-5697338ec9f2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w"
Jul 10 15:56:22.393883 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.393786    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ebf7138d-5fef-4905-9e1f-64c011efc16d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-bdtkv\" (UID: \"ebf7138d-5fef-4905-9e1f-64c011efc16d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bdtkv"
Jul 10 15:56:22.402734 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.402699    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w"]
Jul 10 15:56:22.458075 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.458015    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84cd795589-bb48p"]
Jul 10 15:56:22.460356 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.460326    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.477311 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.477275    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84cd795589-bb48p"]
Jul 10 15:56:22.495013 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.494968    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpl2q\" (UniqueName: \"kubernetes.io/projected/97be136b-397f-4d36-a5b5-5697338ec9f2-kube-api-access-gpl2q\") pod \"managed-serviceaccount-addon-agent-5c7b67f46d-qh46w\" (UID: \"97be136b-397f-4d36-a5b5-5697338ec9f2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w"
Jul 10 15:56:22.495200 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.495026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ebf7138d-5fef-4905-9e1f-64c011efc16d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bdtkv\" (UID: \"ebf7138d-5fef-4905-9e1f-64c011efc16d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bdtkv"
Jul 10 15:56:22.495200 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.495066    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/97be136b-397f-4d36-a5b5-5697338ec9f2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c7b67f46d-qh46w\" (UID: \"97be136b-397f-4d36-a5b5-5697338ec9f2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w"
Jul 10 15:56:22.495200 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.495099    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ebf7138d-5fef-4905-9e1f-64c011efc16d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-bdtkv\" (UID: \"ebf7138d-5fef-4905-9e1f-64c011efc16d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bdtkv"
Jul 10 15:56:22.495818 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.495791    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ebf7138d-5fef-4905-9e1f-64c011efc16d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-bdtkv\" (UID: \"ebf7138d-5fef-4905-9e1f-64c011efc16d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bdtkv"
Jul 10 15:56:22.502558 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.502530    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ebf7138d-5fef-4905-9e1f-64c011efc16d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bdtkv\" (UID: \"ebf7138d-5fef-4905-9e1f-64c011efc16d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bdtkv"
Jul 10 15:56:22.504565 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.504512    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/97be136b-397f-4d36-a5b5-5697338ec9f2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c7b67f46d-qh46w\" (UID: \"97be136b-397f-4d36-a5b5-5697338ec9f2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w"
Jul 10 15:56:22.505445 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.505397    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpl2q\" (UniqueName: \"kubernetes.io/projected/97be136b-397f-4d36-a5b5-5697338ec9f2-kube-api-access-gpl2q\") pod \"managed-serviceaccount-addon-agent-5c7b67f46d-qh46w\" (UID: \"97be136b-397f-4d36-a5b5-5697338ec9f2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w"
Jul 10 15:56:22.596261 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.596165    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc9pv\" (UniqueName: \"kubernetes.io/projected/bd67177f-3b05-40fc-a5ad-7c23c53a9253-kube-api-access-hc9pv\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.596261 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.596212    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd67177f-3b05-40fc-a5ad-7c23c53a9253-installation-pull-secrets\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.596261 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.596242    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd67177f-3b05-40fc-a5ad-7c23c53a9253-registry-tls\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.596548 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.596304    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd67177f-3b05-40fc-a5ad-7c23c53a9253-registry-certificates\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.596548 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.596338    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd67177f-3b05-40fc-a5ad-7c23c53a9253-image-registry-private-configuration\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.596548 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.596359    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd67177f-3b05-40fc-a5ad-7c23c53a9253-bound-sa-token\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.596548 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.596377    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd67177f-3b05-40fc-a5ad-7c23c53a9253-trusted-ca\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.596548 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.596424    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd67177f-3b05-40fc-a5ad-7c23c53a9253-ca-trust-extracted\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.623858 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.623827    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-bdtkv"
Jul 10 15:56:22.697317 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.697287    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd67177f-3b05-40fc-a5ad-7c23c53a9253-trusted-ca\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.697514 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.697334    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd67177f-3b05-40fc-a5ad-7c23c53a9253-ca-trust-extracted\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.697514 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.697394    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9pv\" (UniqueName: \"kubernetes.io/projected/bd67177f-3b05-40fc-a5ad-7c23c53a9253-kube-api-access-hc9pv\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.697514 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.697443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd67177f-3b05-40fc-a5ad-7c23c53a9253-installation-pull-secrets\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.697514 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.697479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd67177f-3b05-40fc-a5ad-7c23c53a9253-registry-tls\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.697697 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.697529    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd67177f-3b05-40fc-a5ad-7c23c53a9253-registry-certificates\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.697697 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.697562    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd67177f-3b05-40fc-a5ad-7c23c53a9253-image-registry-private-configuration\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.697697 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.697606    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd67177f-3b05-40fc-a5ad-7c23c53a9253-bound-sa-token\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.697825 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.697811    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bd67177f-3b05-40fc-a5ad-7c23c53a9253-ca-trust-extracted\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.699027 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.698278    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bd67177f-3b05-40fc-a5ad-7c23c53a9253-trusted-ca\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.699027 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.698625    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bd67177f-3b05-40fc-a5ad-7c23c53a9253-registry-certificates\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.700825 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.700785    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bd67177f-3b05-40fc-a5ad-7c23c53a9253-registry-tls\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.701305 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.701283    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bd67177f-3b05-40fc-a5ad-7c23c53a9253-image-registry-private-configuration\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.701586 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.701559    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bd67177f-3b05-40fc-a5ad-7c23c53a9253-installation-pull-secrets\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.711635 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.711608    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd67177f-3b05-40fc-a5ad-7c23c53a9253-bound-sa-token\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.712034 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.711791    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc9pv\" (UniqueName: \"kubernetes.io/projected/bd67177f-3b05-40fc-a5ad-7c23c53a9253-kube-api-access-hc9pv\") pod \"image-registry-84cd795589-bb48p\" (UID: \"bd67177f-3b05-40fc-a5ad-7c23c53a9253\") " pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:22.719750 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.719214    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59786c5dc6-qhtdf" event={"ID":"ef2cf27d-6566-4821-9282-c222c9b35c50","Type":"ContainerStarted","Data":"f7b92c4ceacc8491abb9de82476be45febfa352a6b895860111a5f303e830c82"}
Jul 10 15:56:22.719750 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.719275    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59786c5dc6-qhtdf" event={"ID":"ef2cf27d-6566-4821-9282-c222c9b35c50","Type":"ContainerStarted","Data":"86f7ad75ab5b912f1f07011f3d8989d4e8f9bac436f26fa2b4cb1f5ee758bb53"}
Jul 10 15:56:22.721103 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.721044    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h" event={"ID":"949e1ae5-ccf7-479c-a0f1-526bd9a254bf","Type":"ContainerStarted","Data":"1fa4636a1d49f2c84d5426d4cd5cfca135035c84e09d67c161ef599510cfd783"}
Jul 10 15:56:22.749794 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.749171    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-59786c5dc6-qhtdf" podStartSLOduration=33.7491499 podStartE2EDuration="33.7491499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:22.744817802 +0000 UTC m=+69.888351643" watchObservedRunningTime="2026-07-10 15:56:22.7491499 +0000 UTC m=+69.892683719"
Jul 10 15:56:22.770633 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:22.770593    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:23.266180 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:23.266142    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-52w6s\""
Jul 10 15:56:23.282282 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:23.282249    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 10 15:56:23.289594 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:23.289564    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w"
Jul 10 15:56:23.584641 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:23.584606    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:56:23.587741 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:23.587511    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:56:23.725255 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:23.725221    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:56:23.726701 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:23.726678    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-59786c5dc6-qhtdf"
Jul 10 15:56:25.259813 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.259775    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w"]
Jul 10 15:56:25.267539 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.266138    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-bdtkv"]
Jul 10 15:56:25.299220 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.299145    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84cd795589-bb48p"]
Jul 10 15:56:25.302845 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:56:25.302745    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd67177f_3b05_40fc_a5ad_7c23c53a9253.slice/crio-2d82f027b54967262e5f4ed12d24a21a10df5f8b3d515af2637a61bb575d857f WatchSource:0}: Error finding container 2d82f027b54967262e5f4ed12d24a21a10df5f8b3d515af2637a61bb575d857f: Status 404 returned error can't find the container with id 2d82f027b54967262e5f4ed12d24a21a10df5f8b3d515af2637a61bb575d857f
Jul 10 15:56:25.734193 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.734156    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l5nkc" event={"ID":"16d3e229-e008-438f-8f5a-13d9364e5b7b","Type":"ContainerStarted","Data":"70fb2f52a37a94d7c05503d2feff90157c02fed776882bfe6e36feea6d0729eb"}
Jul 10 15:56:25.735803 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.735774    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xbfbt" event={"ID":"25a66fdf-5eef-4fd3-b7b6-369254a11b3f","Type":"ContainerStarted","Data":"87dedfe8dea6df7af3438e6b5edd09d2e1c0be8e498317c73764ecfdd714ad87"}
Jul 10 15:56:25.735976 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.735842    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:56:25.737851 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.737820    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z8vr5" event={"ID":"a6f5c915-40a8-4894-b83a-01985ca022fb","Type":"ContainerStarted","Data":"c96b3124572cd6ed3cc50f298da21edaace9fcd22948d9861242060889cabb29"}
Jul 10 15:56:25.737988 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.737880    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:56:25.737988 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.737906    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z8vr5" event={"ID":"a6f5c915-40a8-4894-b83a-01985ca022fb","Type":"ContainerStarted","Data":"23ef988eb6b25695f24c8eec047814cde9b48ab71ca60fcf1f01624e8e0eddef"}
Jul 10 15:56:25.738985 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.738964    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w" event={"ID":"97be136b-397f-4d36-a5b5-5697338ec9f2","Type":"ContainerStarted","Data":"5bf0d64e884a08ecc82dfab10a0e498210bdd839875f5b4a7e6cf3daf4d77b78"}
Jul 10 15:56:25.740255 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.740228    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-bdtkv" event={"ID":"ebf7138d-5fef-4905-9e1f-64c011efc16d","Type":"ContainerStarted","Data":"679cfdc88dd07f3961d09cd3237f6bb4ac5bc95fc8dd5f97d778842d88ed07af"}
Jul 10 15:56:25.741887 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.741865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h" event={"ID":"949e1ae5-ccf7-479c-a0f1-526bd9a254bf","Type":"ContainerStarted","Data":"1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa"}
Jul 10 15:56:25.741990 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.741946    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:56:25.743816 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.743788    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84cd795589-bb48p" event={"ID":"bd67177f-3b05-40fc-a5ad-7c23c53a9253","Type":"ContainerStarted","Data":"085f6658d4fa7917938493303b67ceb638a91385a0537978fd70e242f5f79ae8"}
Jul 10 15:56:25.743926 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.743827    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84cd795589-bb48p" event={"ID":"bd67177f-3b05-40fc-a5ad-7c23c53a9253","Type":"ContainerStarted","Data":"2d82f027b54967262e5f4ed12d24a21a10df5f8b3d515af2637a61bb575d857f"}
Jul 10 15:56:25.752995 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.752931    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-l5nkc" podStartSLOduration=67.216080583 podStartE2EDuration="1m12.752914699s" totalImagesPullingTime="5.536834116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:25.752608148 +0000 UTC m=+72.896141989" watchObservedRunningTime="2026-07-10 15:56:25.752914699 +0000 UTC m=+72.896448516"
Jul 10 15:56:25.775854 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.775780    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84cd795589-bb48p" podStartSLOduration=3.7757639039999997 podStartE2EDuration="3.775763904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:56:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:25.774380013 +0000 UTC m=+72.917913852" watchObservedRunningTime="2026-07-10 15:56:25.775763904 +0000 UTC m=+72.919297721"
Jul 10 15:56:25.801610 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.801549    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xbfbt" podStartSLOduration=67.432997582 podStartE2EDuration="1m12.801529356s" totalImagesPullingTime="5.368531774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:25.800222345 +0000 UTC m=+72.943756198" watchObservedRunningTime="2026-07-10 15:56:25.801529356 +0000 UTC m=+72.945063196"
Jul 10 15:56:25.823549 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.823482    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-z8vr5" podStartSLOduration=33.308932812 podStartE2EDuration="36.823460805s" totalImagesPullingTime="3.514527993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:25.822118855 +0000 UTC m=+72.965652695" watchObservedRunningTime="2026-07-10 15:56:25.823460805 +0000 UTC m=+72.966994649"
Jul 10 15:56:25.848321 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:25.848247    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h" podStartSLOduration=36.848224034 podStartE2EDuration="36.848224034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:25.84749899 +0000 UTC m=+72.991032832" watchObservedRunningTime="2026-07-10 15:56:25.848224034 +0000 UTC m=+72.991757876"
Jul 10 15:56:26.748424 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:26.748336    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:28.049915 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:28.049885    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:56:28.052242 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:28.052222    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/194db3c8-57e0-4bec-bef7-c2ef3ca7fce7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sjb7f\" (UID: \"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7\") " pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:56:28.297186 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:28.297099    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6kc2\""
Jul 10 15:56:28.304634 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:28.304610    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sjb7f"
Jul 10 15:56:28.609439 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:28.609394    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sjb7f"]
Jul 10 15:56:28.611769 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:56:28.611739    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod194db3c8_57e0_4bec_bef7_c2ef3ca7fce7.slice/crio-91c09ab3a7fbacc1d636110d56fd7612998f72ec4c32e92e181e1b0b3d277e4e WatchSource:0}: Error finding container 91c09ab3a7fbacc1d636110d56fd7612998f72ec4c32e92e181e1b0b3d277e4e: Status 404 returned error can't find the container with id 91c09ab3a7fbacc1d636110d56fd7612998f72ec4c32e92e181e1b0b3d277e4e
Jul 10 15:56:28.756120 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:28.756078    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w" event={"ID":"97be136b-397f-4d36-a5b5-5697338ec9f2","Type":"ContainerStarted","Data":"e016873e0af767a36a06f05cf6e6a20506d7bad7417e976d725046f0feb5a0b8"}
Jul 10 15:56:28.757723 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:28.757694    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-bdtkv" event={"ID":"ebf7138d-5fef-4905-9e1f-64c011efc16d","Type":"ContainerStarted","Data":"9a83ba595e24e1fce49e27ebaeef51cd50e543fc67f8791cd0026bb93ce3b2fd"}
Jul 10 15:56:28.758831 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:28.758810    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjb7f" event={"ID":"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7","Type":"ContainerStarted","Data":"91c09ab3a7fbacc1d636110d56fd7612998f72ec4c32e92e181e1b0b3d277e4e"}
Jul 10 15:56:28.779243 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:28.779196    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c7b67f46d-qh46w" podStartSLOduration=3.615203591 podStartE2EDuration="6.779181565s" totalImagesPullingTime="3.163977974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:28.77814868 +0000 UTC m=+75.921682522" watchObservedRunningTime="2026-07-10 15:56:28.779181565 +0000 UTC m=+75.922715382"
Jul 10 15:56:28.800381 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:28.800327    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-bdtkv" podStartSLOduration=4.075511258 podStartE2EDuration="6.800311452s" totalImagesPullingTime="2.724800194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:28.798915848 +0000 UTC m=+75.942449688" watchObservedRunningTime="2026-07-10 15:56:28.800311452 +0000 UTC m=+75.943845290"
Jul 10 15:56:32.779939 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:32.779904    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjb7f" event={"ID":"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7","Type":"ContainerStarted","Data":"9ffbf6e47d4c8db035a1cccf838728078484229c71d25a2a099709d943e07616"}
Jul 10 15:56:33.786820 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:33.786679    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjb7f" event={"ID":"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7","Type":"ContainerStarted","Data":"182009d82a5ffc8d22a2edfb4fe05240906ab51289fade145fdfa6b9c03f8262"}
Jul 10 15:56:35.750326 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:35.750287    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-z8vr5"
Jul 10 15:56:36.799809 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:36.799769    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sjb7f" event={"ID":"194db3c8-57e0-4bec-bef7-c2ef3ca7fce7","Type":"ContainerStarted","Data":"d2ae2f0a0250a0390e190272d23e7e933837bdac33e3815d74b9fc72b985dcf4"}
Jul 10 15:56:36.831164 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:36.831114    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-sjb7f" podStartSLOduration=33.759561194 podStartE2EDuration="40.831098291s" totalImagesPullingTime="7.071537097s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:36.829838465 +0000 UTC m=+83.973372305" watchObservedRunningTime="2026-07-10 15:56:36.831098291 +0000 UTC m=+83.974632129"
Jul 10 15:56:37.448388 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.448354    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cj2vz"]
Jul 10 15:56:37.450959 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.450936    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.456296 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.456259    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 15:56:37.456296 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.456260    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 15:56:37.457328 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.457302    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nn7fg\""
Jul 10 15:56:37.457328 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.457330    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 15:56:37.457543 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.457423    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 15:56:37.457543 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.457448    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 15:56:37.458009 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.457705    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 15:56:37.529615 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.529583    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.529795 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.529621    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7fv2\" (UniqueName: \"kubernetes.io/projected/e68dd708-b1d0-48b8-8926-7824da62145c-kube-api-access-h7fv2\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.529795 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.529646    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e68dd708-b1d0-48b8-8926-7824da62145c-root\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.529795 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.529743    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-accelerators-collector-config\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.529795 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.529773    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e68dd708-b1d0-48b8-8926-7824da62145c-sys\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.529964 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.529803    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-textfile\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.529964 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.529822    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e68dd708-b1d0-48b8-8926-7824da62145c-metrics-client-ca\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.529964 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.529864    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-tls\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.529964 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.529896    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-wtmp\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.630757 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.630718    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e68dd708-b1d0-48b8-8926-7824da62145c-root\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.630987 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.630767    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-accelerators-collector-config\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.630987 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.630829    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e68dd708-b1d0-48b8-8926-7824da62145c-root\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.630987 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.630880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e68dd708-b1d0-48b8-8926-7824da62145c-sys\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.630987 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.630919    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-textfile\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.630987 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.630937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e68dd708-b1d0-48b8-8926-7824da62145c-metrics-client-ca\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.630987 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.630977    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-tls\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.631214 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.630990    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e68dd708-b1d0-48b8-8926-7824da62145c-sys\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.631214 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.630999    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-wtmp\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.631214 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.631054    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.631214 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.631081    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7fv2\" (UniqueName: \"kubernetes.io/projected/e68dd708-b1d0-48b8-8926-7824da62145c-kube-api-access-h7fv2\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.631214 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:37.631086    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 10 15:56:37.631214 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:37.631198    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-tls podName:e68dd708-b1d0-48b8-8926-7824da62145c nodeName:}" failed. No retries permitted until 2026-07-10 15:56:38.131178565 +0000 UTC m=+85.274712397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-tls") pod "node-exporter-cj2vz" (UID: "e68dd708-b1d0-48b8-8926-7824da62145c") : secret "node-exporter-tls" not found
Jul 10 15:56:37.631214 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.631125    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-wtmp\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.631480 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.631297    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-textfile\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.631480 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.631367    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-accelerators-collector-config\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.631480 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.631473    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e68dd708-b1d0-48b8-8926-7824da62145c-metrics-client-ca\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.633566 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.633545    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:37.647596 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:37.647565    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7fv2\" (UniqueName: \"kubernetes.io/projected/e68dd708-b1d0-48b8-8926-7824da62145c-kube-api-access-h7fv2\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:38.135623 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:38.135583    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-tls\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:38.138076 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:38.138049    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e68dd708-b1d0-48b8-8926-7824da62145c-node-exporter-tls\") pod \"node-exporter-cj2vz\" (UID: \"e68dd708-b1d0-48b8-8926-7824da62145c\") " pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:38.360857 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:38.360821    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cj2vz"
Jul 10 15:56:38.371163 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:56:38.371126    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode68dd708_b1d0_48b8_8926_7824da62145c.slice/crio-855a1cd41152343c171b78ce25952c95098e704ae52362094014757146373759 WatchSource:0}: Error finding container 855a1cd41152343c171b78ce25952c95098e704ae52362094014757146373759: Status 404 returned error can't find the container with id 855a1cd41152343c171b78ce25952c95098e704ae52362094014757146373759
Jul 10 15:56:38.808167 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:38.808125    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cj2vz" event={"ID":"e68dd708-b1d0-48b8-8926-7824da62145c","Type":"ContainerStarted","Data":"855a1cd41152343c171b78ce25952c95098e704ae52362094014757146373759"}
Jul 10 15:56:39.812973 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:39.812942    2567 generic.go:356] "Generic (PLEG): container finished" podID="e68dd708-b1d0-48b8-8926-7824da62145c" containerID="1209d93528dc9486358cc4fb5563dc7fa432c4bd7d2fbe427aa8df83646d1a39" exitCode=0
Jul 10 15:56:39.813358 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:39.813016    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cj2vz" event={"ID":"e68dd708-b1d0-48b8-8926-7824da62145c","Type":"ContainerDied","Data":"1209d93528dc9486358cc4fb5563dc7fa432c4bd7d2fbe427aa8df83646d1a39"}
Jul 10 15:56:40.818483 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:40.818446    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cj2vz" event={"ID":"e68dd708-b1d0-48b8-8926-7824da62145c","Type":"ContainerStarted","Data":"57f351ae4e3354556950ac5fcd972cb87eb781eaa11574afd150c711307d75c1"}
Jul 10 15:56:40.819089 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:40.818535    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cj2vz" event={"ID":"e68dd708-b1d0-48b8-8926-7824da62145c","Type":"ContainerStarted","Data":"40e2b57bef0c6a2448547fe26f4626357a3ce376ac2a1d13ac3e56c4002af1fa"}
Jul 10 15:56:40.857852 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:40.857804    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cj2vz" podStartSLOduration=2.973847303 podStartE2EDuration="3.857786657s" totalImagesPullingTime="810.974783ms" totalInitContainerRuntime="72.964571ms" isStatefulPod=true podCreationTimestamp="2026-07-10 15:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:40.857039639 +0000 UTC m=+88.000573490" watchObservedRunningTime="2026-07-10 15:56:40.857786657 +0000 UTC m=+88.001320495"
Jul 10 15:56:42.196870 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.196839    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"]
Jul 10 15:56:42.198891 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.198874    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"
Jul 10 15:56:42.201827 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.201800    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 10 15:56:42.201941 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.201856    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4mjkc\""
Jul 10 15:56:42.210015 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.209985    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"]
Jul 10 15:56:42.369619 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.369581    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mmvj\" (UniqueName: \"kubernetes.io/projected/faced6ae-ed28-45dd-b9c2-7d86892254f4-kube-api-access-2mmvj\") pod \"monitoring-plugin-5f68df7f58-8lsrg\" (UID: \"faced6ae-ed28-45dd-b9c2-7d86892254f4\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"
Jul 10 15:56:42.369804 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.369640    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/faced6ae-ed28-45dd-b9c2-7d86892254f4-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-8lsrg\" (UID: \"faced6ae-ed28-45dd-b9c2-7d86892254f4\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"
Jul 10 15:56:42.470747 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.470644    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mmvj\" (UniqueName: \"kubernetes.io/projected/faced6ae-ed28-45dd-b9c2-7d86892254f4-kube-api-access-2mmvj\") pod \"monitoring-plugin-5f68df7f58-8lsrg\" (UID: \"faced6ae-ed28-45dd-b9c2-7d86892254f4\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"
Jul 10 15:56:42.470747 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.470695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/faced6ae-ed28-45dd-b9c2-7d86892254f4-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-8lsrg\" (UID: \"faced6ae-ed28-45dd-b9c2-7d86892254f4\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"
Jul 10 15:56:42.473355 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.473322    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/faced6ae-ed28-45dd-b9c2-7d86892254f4-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-8lsrg\" (UID: \"faced6ae-ed28-45dd-b9c2-7d86892254f4\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"
Jul 10 15:56:42.478969 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.478940    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mmvj\" (UniqueName: \"kubernetes.io/projected/faced6ae-ed28-45dd-b9c2-7d86892254f4-kube-api-access-2mmvj\") pod \"monitoring-plugin-5f68df7f58-8lsrg\" (UID: \"faced6ae-ed28-45dd-b9c2-7d86892254f4\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"
Jul 10 15:56:42.507979 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.507944    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"
Jul 10 15:56:42.643110 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.643072    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"]
Jul 10 15:56:42.648805 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:56:42.648743    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaced6ae_ed28_45dd_b9c2_7d86892254f4.slice/crio-e91c59fdfb7e87e870503b912b86af81a684975e2256c4613fc7d3123793792c WatchSource:0}: Error finding container e91c59fdfb7e87e870503b912b86af81a684975e2256c4613fc7d3123793792c: Status 404 returned error can't find the container with id e91c59fdfb7e87e870503b912b86af81a684975e2256c4613fc7d3123793792c
Jul 10 15:56:42.827277 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:42.827180    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg" event={"ID":"faced6ae-ed28-45dd-b9c2-7d86892254f4","Type":"ContainerStarted","Data":"e91c59fdfb7e87e870503b912b86af81a684975e2256c4613fc7d3123793792c"}
Jul 10 15:56:43.722948 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.722902    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 15:56:43.726393 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.726359    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.729183 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.728898    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 15:56:43.729183 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.728898    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 15:56:43.729183 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.729051    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 15:56:43.730988 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.729823    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 15:56:43.730988 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.729912    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 15:56:43.730988 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.730151    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-r5npt\""
Jul 10 15:56:43.730988 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.730585    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 15:56:43.730988 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.730830    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 15:56:43.731602 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.731581    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 15:56:43.731770 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.731749    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 15:56:43.732074 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.732033    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7eev8fbh72mla\""
Jul 10 15:56:43.732170 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.732075    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 15:56:43.732170 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.732137    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 15:56:43.732170 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.732148    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 15:56:43.732359 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.732342    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 15:56:43.732522 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.732504    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 15:56:43.733816 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.733799    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 15:56:43.742012 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.741986    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 15:56:43.883161 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883123    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.883161 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883168    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qvqq\" (UniqueName: \"kubernetes.io/projected/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-kube-api-access-8qvqq\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.883437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883185    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.883437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883217    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.883437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883241    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.883437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883265    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.883437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883300    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.883437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883317    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-config\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.883437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883337    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.883437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883375    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.883437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883398    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.883437 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883437    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.884273 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883491    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.884273 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883539    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.884273 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-web-config\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.884273 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883590    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-config-out\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.884273 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883615    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.884273 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883644    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.884273 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883690    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.884273 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.883715    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985165 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985165 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985120    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985165 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985140    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985481 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985173    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985481 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985201    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qvqq\" (UniqueName: \"kubernetes.io/projected/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-kube-api-access-8qvqq\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985481 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985481 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985260    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985481 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985291    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985481 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985318    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985481 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985353    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985481 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985382    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-config\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985481 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985431    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985481 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985471    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985976 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985493    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985976 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985521    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985976 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985550    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985976 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985592    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985976 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985619    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-web-config\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985976 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985652    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-config-out\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985976 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985676    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.985976 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.985968    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.986466 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.986175    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.986466 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.986311    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.986466 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.986358    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.987643 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.987090    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.988652 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.988626    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-config\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.989515 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.989490    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.991467 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.989921    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.991467 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.990305    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.991467 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.990505    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-web-config\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.991467 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.990748    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.991467 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.991382    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.991866 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.991823    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.991866 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.991858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.992475 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.992451    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.992930 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.992905    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.993012 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.992906    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.993076 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.993048    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-config-out\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.993290 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.993266    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:43.998755 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:43.998714    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qvqq\" (UniqueName: \"kubernetes.io/projected/e0e15cb7-dbbd-4d21-9625-4a4cf716c296-kube-api-access-8qvqq\") pod \"prometheus-k8s-0\" (UID: \"e0e15cb7-dbbd-4d21-9625-4a4cf716c296\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:44.038749 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:44.038709    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:44.431787 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:44.431756    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 15:56:44.435054 ip-10-0-135-156 kubenswrapper[2567]: W0710 15:56:44.435027    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e15cb7_dbbd_4d21_9625_4a4cf716c296.slice/crio-9ef5b331fe81c328fdeed62e79e4593c3a2a666c7fa646b53011373247ce118b WatchSource:0}: Error finding container 9ef5b331fe81c328fdeed62e79e4593c3a2a666c7fa646b53011373247ce118b: Status 404 returned error can't find the container with id 9ef5b331fe81c328fdeed62e79e4593c3a2a666c7fa646b53011373247ce118b
Jul 10 15:56:44.835206 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:44.835170    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e0e15cb7-dbbd-4d21-9625-4a4cf716c296","Type":"ContainerStarted","Data":"9ef5b331fe81c328fdeed62e79e4593c3a2a666c7fa646b53011373247ce118b"}
Jul 10 15:56:44.836522 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:44.836497    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg" event={"ID":"faced6ae-ed28-45dd-b9c2-7d86892254f4","Type":"ContainerStarted","Data":"c379aac0848a3eacced82743d12a567da2248cbe99412da8a3129280770a0507"}
Jul 10 15:56:44.836808 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:44.836777    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"
Jul 10 15:56:44.841706 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:44.841684    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg"
Jul 10 15:56:44.854497 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:44.854453    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-8lsrg" podStartSLOduration=1.193034691 podStartE2EDuration="2.854432615s" totalImagesPullingTime="1.661397924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:56:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:44.853139896 +0000 UTC m=+91.996673736" watchObservedRunningTime="2026-07-10 15:56:44.854432615 +0000 UTC m=+91.997966467"
Jul 10 15:56:45.747496 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:45.747470    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:56:45.842045 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:45.842013    2567 generic.go:356] "Generic (PLEG): container finished" podID="e0e15cb7-dbbd-4d21-9625-4a4cf716c296" containerID="0215e8583adc351c7ed35668bcb8eb75cb4b028c40e576618728c80261ede124" exitCode=0
Jul 10 15:56:45.842389 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:45.842107    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e0e15cb7-dbbd-4d21-9625-4a4cf716c296","Type":"ContainerDied","Data":"0215e8583adc351c7ed35668bcb8eb75cb4b028c40e576618728c80261ede124"}
Jul 10 15:56:47.755619 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:47.755590    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84cd795589-bb48p"
Jul 10 15:56:49.861884 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:49.861846    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e0e15cb7-dbbd-4d21-9625-4a4cf716c296","Type":"ContainerStarted","Data":"5f821c11c2d8626f655688ad941ce407261d9e3625c5d06760f460dd3173c3cb"}
Jul 10 15:56:49.862332 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:49.861932    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e0e15cb7-dbbd-4d21-9625-4a4cf716c296","Type":"ContainerStarted","Data":"d64fe86bb8ed75d1688e7b1427801b639368127d2d870698e6034b9983275c26"}
Jul 10 15:56:50.764720 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:50.764659    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h" podUID="949e1ae5-ccf7-479c-a0f1-526bd9a254bf" containerName="registry" containerID="cri-o://1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa" gracePeriod=30
Jul 10 15:56:51.047553 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.047523    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:56:51.152575 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.152541    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcrh6\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-kube-api-access-wcrh6\") pod \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") "
Jul 10 15:56:51.152691 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.152616    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls\") pod \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") "
Jul 10 15:56:51.152691 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.152660    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-trusted-ca\") pod \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") "
Jul 10 15:56:51.152780 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.152687    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-certificates\") pod \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") "
Jul 10 15:56:51.152780 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.152721    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-bound-sa-token\") pod \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") "
Jul 10 15:56:51.152881 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.152751    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-installation-pull-secrets\") pod \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") "
Jul 10 15:56:51.152934 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.152880    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-ca-trust-extracted\") pod \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") "
Jul 10 15:56:51.152934 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.152922    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-image-registry-private-configuration\") pod \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\" (UID: \"949e1ae5-ccf7-479c-a0f1-526bd9a254bf\") "
Jul 10 15:56:51.153117 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.153082    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "949e1ae5-ccf7-479c-a0f1-526bd9a254bf" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:56:51.153208 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.153193    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-trusted-ca\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 15:56:51.154931 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.153574    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "949e1ae5-ccf7-479c-a0f1-526bd9a254bf" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 15:56:51.155102 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.155048    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-kube-api-access-wcrh6" (OuterVolumeSpecName: "kube-api-access-wcrh6") pod "949e1ae5-ccf7-479c-a0f1-526bd9a254bf" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf"). InnerVolumeSpecName "kube-api-access-wcrh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:56:51.155958 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.155899    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "949e1ae5-ccf7-479c-a0f1-526bd9a254bf" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:56:51.156217 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.156196    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "949e1ae5-ccf7-479c-a0f1-526bd9a254bf" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:56:51.156317 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.156303    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "949e1ae5-ccf7-479c-a0f1-526bd9a254bf" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 15:56:51.157237 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.157218    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "949e1ae5-ccf7-479c-a0f1-526bd9a254bf" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 15:56:51.163131 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.163104    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "949e1ae5-ccf7-479c-a0f1-526bd9a254bf" (UID: "949e1ae5-ccf7-479c-a0f1-526bd9a254bf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 15:56:51.254267 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.254224    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcrh6\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-kube-api-access-wcrh6\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 15:56:51.254267 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.254257    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-tls\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 15:56:51.254267 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.254267    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-registry-certificates\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 15:56:51.254549 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.254280    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-bound-sa-token\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 15:56:51.254549 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.254289    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-installation-pull-secrets\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 15:56:51.254549 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.254300    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-ca-trust-extracted\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 15:56:51.254549 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.254312    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/949e1ae5-ccf7-479c-a0f1-526bd9a254bf-image-registry-private-configuration\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 15:56:51.870711 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.870680    2567 generic.go:356] "Generic (PLEG): container finished" podID="949e1ae5-ccf7-479c-a0f1-526bd9a254bf" containerID="1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa" exitCode=0
Jul 10 15:56:51.870877 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.870742    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"
Jul 10 15:56:51.870877 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.870773    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h" event={"ID":"949e1ae5-ccf7-479c-a0f1-526bd9a254bf","Type":"ContainerDied","Data":"1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa"}
Jul 10 15:56:51.870877 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.870861    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68ff9d4cf8-f9b9h" event={"ID":"949e1ae5-ccf7-479c-a0f1-526bd9a254bf","Type":"ContainerDied","Data":"1fa4636a1d49f2c84d5426d4cd5cfca135035c84e09d67c161ef599510cfd783"}
Jul 10 15:56:51.871066 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.870888    2567 scope.go:117] "RemoveContainer" containerID="1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa"
Jul 10 15:56:51.874695 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.874668    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e0e15cb7-dbbd-4d21-9625-4a4cf716c296","Type":"ContainerStarted","Data":"006e7cae569532f9aa5ece004d2cceb37e11bb65b9751ca13be091a070c6d760"}
Jul 10 15:56:51.874818 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.874741    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e0e15cb7-dbbd-4d21-9625-4a4cf716c296","Type":"ContainerStarted","Data":"29c4492a1b620da7b5bf422aba4f0445cbde60e95472c929862592d40045b8ea"}
Jul 10 15:56:51.874818 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.874781    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e0e15cb7-dbbd-4d21-9625-4a4cf716c296","Type":"ContainerStarted","Data":"b4c567ffa817a3e6a95484bb39b9940ee80df5c5cd11817f03e0ea4fb0f2c885"}
Jul 10 15:56:51.874970 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.874822    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e0e15cb7-dbbd-4d21-9625-4a4cf716c296","Type":"ContainerStarted","Data":"28448d5fac9ca606016e552148f4702c5eb41a4dd9cd06bb19ecd826dbf63aa1"}
Jul 10 15:56:51.879900 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.879879    2567 scope.go:117] "RemoveContainer" containerID="1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa"
Jul 10 15:56:51.880226 ip-10-0-135-156 kubenswrapper[2567]: E0710 15:56:51.880205    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa\": container with ID starting with 1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa not found: ID does not exist" containerID="1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa"
Jul 10 15:56:51.880279 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.880236    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa"} err="failed to get container status \"1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa\": rpc error: code = NotFound desc = could not find container \"1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa\": container with ID starting with 1f742201e50914cef19c1529bb0c7da0c336dee66daa654b746837e889d42dfa not found: ID does not exist"
Jul 10 15:56:51.888673 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.888641    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"]
Jul 10 15:56:51.901554 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.901524    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-68ff9d4cf8-f9b9h"]
Jul 10 15:56:51.926822 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:51.926770    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.764836125 podStartE2EDuration="8.926752852s" totalImagesPullingTime="6.141585979s" totalInitContainerRuntime="20.330748ms" isStatefulPod=false podCreationTimestamp="2026-07-10 15:56:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:51.925951827 +0000 UTC m=+99.069485667" watchObservedRunningTime="2026-07-10 15:56:51.926752852 +0000 UTC m=+99.070286752"
Jul 10 15:56:53.404579 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:53.404548    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="949e1ae5-ccf7-479c-a0f1-526bd9a254bf" path="/var/lib/kubelet/pods/949e1ae5-ccf7-479c-a0f1-526bd9a254bf/volumes"
Jul 10 15:56:54.039335 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:54.039304    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:56:56.750467 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:56:56.750436    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xbfbt"
Jul 10 15:57:15.964905 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:57:15.964869    2567 generic.go:356] "Generic (PLEG): container finished" podID="4415dbe9-f215-4e32-b06a-f3fe0affea2c" containerID="bd73e462bd63a1d205bbe036e56936a15cf6eb8b8bc498bbc296b2c8081df2a2" exitCode=0
Jul 10 15:57:15.965368 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:57:15.964913    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-gphds" event={"ID":"4415dbe9-f215-4e32-b06a-f3fe0affea2c","Type":"ContainerDied","Data":"bd73e462bd63a1d205bbe036e56936a15cf6eb8b8bc498bbc296b2c8081df2a2"}
Jul 10 15:57:15.965368 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:57:15.965247    2567 scope.go:117] "RemoveContainer" containerID="bd73e462bd63a1d205bbe036e56936a15cf6eb8b8bc498bbc296b2c8081df2a2"
Jul 10 15:57:16.971077 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:57:16.971042    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-gphds" event={"ID":"4415dbe9-f215-4e32-b06a-f3fe0affea2c","Type":"ContainerStarted","Data":"a6c900d4129b09f3955997779edff8c82fbc80728e07ddc6d8e5dcbf2f940072"}
Jul 10 15:57:26.005753 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:57:26.005717    2567 generic.go:356] "Generic (PLEG): container finished" podID="365169d4-d476-40f7-8ba9-3da44f46edde" containerID="407db4628629e577d4295aa477d1fc8a52ef850a403622bcdc741abb17a19fa2" exitCode=0
Jul 10 15:57:26.006180 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:57:26.005791    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d" event={"ID":"365169d4-d476-40f7-8ba9-3da44f46edde","Type":"ContainerDied","Data":"407db4628629e577d4295aa477d1fc8a52ef850a403622bcdc741abb17a19fa2"}
Jul 10 15:57:26.006180 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:57:26.006132    2567 scope.go:117] "RemoveContainer" containerID="407db4628629e577d4295aa477d1fc8a52ef850a403622bcdc741abb17a19fa2"
Jul 10 15:57:27.011216 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:57:27.011182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-xmk5d" event={"ID":"365169d4-d476-40f7-8ba9-3da44f46edde","Type":"ContainerStarted","Data":"b1dc774916fe1eb9d3b651274253dc46b88ba383fd3f1d9ced8aa043d37827f0"}
Jul 10 15:57:44.039525 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:57:44.039487    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:57:44.056287 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:57:44.056262    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 15:57:44.097355 ip-10-0-135-156 kubenswrapper[2567]: I0710 15:57:44.097331    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 16:00:13.234687 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:13.234660    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 16:00:25.090856 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.090822    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"]
Jul 10 16:00:25.093239 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.091093    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949e1ae5-ccf7-479c-a0f1-526bd9a254bf" containerName="registry"
Jul 10 16:00:25.093239 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.091104    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="949e1ae5-ccf7-479c-a0f1-526bd9a254bf" containerName="registry"
Jul 10 16:00:25.093239 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.091162    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="949e1ae5-ccf7-479c-a0f1-526bd9a254bf" containerName="registry"
Jul 10 16:00:25.094016 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.094000    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.097287 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.097265    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 16:00:25.097714 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.097697    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5fx84\""
Jul 10 16:00:25.098574 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.098560    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 16:00:25.100265 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.100246    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 16:00:25.105727 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.105708    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 16:00:25.130070 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.130038    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"]
Jul 10 16:00:25.196859 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.196819    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa3173d1-73f1-4797-8b79-b80137b2ce7a-tmp\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.197043 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.196874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fa3173d1-73f1-4797-8b79-b80137b2ce7a-manifests\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.197043 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.196910    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa3173d1-73f1-4797-8b79-b80137b2ce7a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.197043 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.196944    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fq94\" (UniqueName: \"kubernetes.io/projected/fa3173d1-73f1-4797-8b79-b80137b2ce7a-kube-api-access-7fq94\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.197043 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.196981    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa3173d1-73f1-4797-8b79-b80137b2ce7a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.297965 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.297930    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fa3173d1-73f1-4797-8b79-b80137b2ce7a-manifests\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.298148 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.297978    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa3173d1-73f1-4797-8b79-b80137b2ce7a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.298148 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.298013    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fq94\" (UniqueName: \"kubernetes.io/projected/fa3173d1-73f1-4797-8b79-b80137b2ce7a-kube-api-access-7fq94\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.298148 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.298066    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa3173d1-73f1-4797-8b79-b80137b2ce7a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.298148 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.298095    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa3173d1-73f1-4797-8b79-b80137b2ce7a-tmp\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.298455 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.298399    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fa3173d1-73f1-4797-8b79-b80137b2ce7a-manifests\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.300628 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.300604    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa3173d1-73f1-4797-8b79-b80137b2ce7a-tmp\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.300765 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.300747    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa3173d1-73f1-4797-8b79-b80137b2ce7a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.300841 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.300819    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa3173d1-73f1-4797-8b79-b80137b2ce7a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.308686 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.308658    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fq94\" (UniqueName: \"kubernetes.io/projected/fa3173d1-73f1-4797-8b79-b80137b2ce7a-kube-api-access-7fq94\") pod \"opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm\" (UID: \"fa3173d1-73f1-4797-8b79-b80137b2ce7a\") " pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.405051 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.404964    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:25.550305 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.550259    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"]
Jul 10 16:00:25.555475 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:00:25.555446    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3173d1_73f1_4797_8b79_b80137b2ce7a.slice/crio-2b700bb8d0aeea35745e768f2e0fa1af69d0d5109474e767644388067727697a WatchSource:0}: Error finding container 2b700bb8d0aeea35745e768f2e0fa1af69d0d5109474e767644388067727697a: Status 404 returned error can't find the container with id 2b700bb8d0aeea35745e768f2e0fa1af69d0d5109474e767644388067727697a
Jul 10 16:00:25.557153 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.557135    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 16:00:25.670795 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:25.670715    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm" event={"ID":"fa3173d1-73f1-4797-8b79-b80137b2ce7a","Type":"ContainerStarted","Data":"2b700bb8d0aeea35745e768f2e0fa1af69d0d5109474e767644388067727697a"}
Jul 10 16:00:28.685792 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:28.685748    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm" event={"ID":"fa3173d1-73f1-4797-8b79-b80137b2ce7a","Type":"ContainerStarted","Data":"802d60aeeb917224f04e9b76137a0c8df876ad5055ebec3a6a1b3341f98a8474"}
Jul 10 16:00:30.695739 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:30.695702    2567 generic.go:356] "Generic (PLEG): container finished" podID="fa3173d1-73f1-4797-8b79-b80137b2ce7a" containerID="802d60aeeb917224f04e9b76137a0c8df876ad5055ebec3a6a1b3341f98a8474" exitCode=0
Jul 10 16:00:30.696157 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:30.695755    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm" event={"ID":"fa3173d1-73f1-4797-8b79-b80137b2ce7a","Type":"ContainerDied","Data":"802d60aeeb917224f04e9b76137a0c8df876ad5055ebec3a6a1b3341f98a8474"}
Jul 10 16:00:31.701770 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:31.701731    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm" event={"ID":"fa3173d1-73f1-4797-8b79-b80137b2ce7a","Type":"ContainerStarted","Data":"040e73f54c3f4f2fb897512afc1d53ff544bdc6af1095bb1221b1075715344a8"}
Jul 10 16:00:31.702253 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:31.701964    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:31.730869 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:31.730822    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm" podStartSLOduration=2.252652243 podStartE2EDuration="6.73080472s" totalImagesPullingTime="3.403799381s" totalInitContainerRuntime="1.074353096s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:00:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:00:31.729113248 +0000 UTC m=+318.872647087" watchObservedRunningTime="2026-07-10 16:00:31.73080472 +0000 UTC m=+318.874338558"
Jul 10 16:00:42.708483 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:42.708451    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm"
Jul 10 16:00:46.375229 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.375192    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n7b4v"]
Jul 10 16:00:46.384495 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.384469    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v"
Jul 10 16:00:46.387656 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.387632    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 10 16:00:46.393258 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.393229    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n7b4v"]
Jul 10 16:00:46.580113 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.580080    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d94hn\" (UniqueName: \"kubernetes.io/projected/3e98b2d3-ce25-46d7-b77d-46377f5dd677-kube-api-access-d94hn\") pod \"model-serving-api-54d89bd5fc-n7b4v\" (UID: \"3e98b2d3-ce25-46d7-b77d-46377f5dd677\") " pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v"
Jul 10 16:00:46.580314 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.580158    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e98b2d3-ce25-46d7-b77d-46377f5dd677-tls-certs\") pod \"model-serving-api-54d89bd5fc-n7b4v\" (UID: \"3e98b2d3-ce25-46d7-b77d-46377f5dd677\") " pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v"
Jul 10 16:00:46.681141 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.681049    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e98b2d3-ce25-46d7-b77d-46377f5dd677-tls-certs\") pod \"model-serving-api-54d89bd5fc-n7b4v\" (UID: \"3e98b2d3-ce25-46d7-b77d-46377f5dd677\") " pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v"
Jul 10 16:00:46.681141 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.681109    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d94hn\" (UniqueName: \"kubernetes.io/projected/3e98b2d3-ce25-46d7-b77d-46377f5dd677-kube-api-access-d94hn\") pod \"model-serving-api-54d89bd5fc-n7b4v\" (UID: \"3e98b2d3-ce25-46d7-b77d-46377f5dd677\") " pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v"
Jul 10 16:00:46.683677 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.683645    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e98b2d3-ce25-46d7-b77d-46377f5dd677-tls-certs\") pod \"model-serving-api-54d89bd5fc-n7b4v\" (UID: \"3e98b2d3-ce25-46d7-b77d-46377f5dd677\") " pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v"
Jul 10 16:00:46.690836 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.690805    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d94hn\" (UniqueName: \"kubernetes.io/projected/3e98b2d3-ce25-46d7-b77d-46377f5dd677-kube-api-access-d94hn\") pod \"model-serving-api-54d89bd5fc-n7b4v\" (UID: \"3e98b2d3-ce25-46d7-b77d-46377f5dd677\") " pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v"
Jul 10 16:00:46.695588 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.695559    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v"
Jul 10 16:00:46.836296 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:46.836268    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n7b4v"]
Jul 10 16:00:46.838921 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:00:46.838891    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e98b2d3_ce25_46d7_b77d_46377f5dd677.slice/crio-b819d42d89c2488884af1efd304f524416046d593787193a85d364830ac66417 WatchSource:0}: Error finding container b819d42d89c2488884af1efd304f524416046d593787193a85d364830ac66417: Status 404 returned error can't find the container with id b819d42d89c2488884af1efd304f524416046d593787193a85d364830ac66417
Jul 10 16:00:47.763555 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:47.763519    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v" event={"ID":"3e98b2d3-ce25-46d7-b77d-46377f5dd677","Type":"ContainerStarted","Data":"b819d42d89c2488884af1efd304f524416046d593787193a85d364830ac66417"}
Jul 10 16:00:48.769313 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:48.769279    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v" event={"ID":"3e98b2d3-ce25-46d7-b77d-46377f5dd677","Type":"ContainerStarted","Data":"5c21229debfced07a00100f08b305f043f575262be4125d7a1fa553c20d829de"}
Jul 10 16:00:48.769773 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:48.769345    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v"
Jul 10 16:00:48.787505 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:48.787450    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v" podStartSLOduration=1.035162367 podStartE2EDuration="2.787429613s" totalImagesPullingTime="1.752267246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:00:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:00:48.786388356 +0000 UTC m=+335.929922195" watchObservedRunningTime="2026-07-10 16:00:48.787429613 +0000 UTC m=+335.930963449"
Jul 10 16:00:52.830544 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.830512    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"]
Jul 10 16:00:52.833694 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.833675    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:52.838858 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.838832    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-5nh6t\""
Jul 10 16:00:52.839462 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.839442    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 16:00:52.852545 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.852519    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"]
Jul 10 16:00:52.924997 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.924957    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:52.924997 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.925002    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9knp\" (UniqueName: \"kubernetes.io/projected/b1a65958-a086-4f31-b544-8aeebcc054e3-kube-api-access-h9knp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:52.925238 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.925051    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:52.925238 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.925073    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:52.925238 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.925164    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:52.925238 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.925201    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:52.925238 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.925222    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:52.925666 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.925259    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1a65958-a086-4f31-b544-8aeebcc054e3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:52.925666 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:52.925297    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.026485 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.026445    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.026485 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.026488    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9knp\" (UniqueName: \"kubernetes.io/projected/b1a65958-a086-4f31-b544-8aeebcc054e3-kube-api-access-h9knp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.026739 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.026518    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.026739 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.026538    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.026739 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.026572    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.026739 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.026593    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.026739 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.026614    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.026739 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.026632    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1a65958-a086-4f31-b544-8aeebcc054e3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.026739 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.026672    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.027083 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.027019    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.027216 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.027187    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.027287 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.027218    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.027472 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.027450    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.027871 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.027850    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1a65958-a086-4f31-b544-8aeebcc054e3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.029074 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.029039    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.029250 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.029228    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.049912 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.049884    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9knp\" (UniqueName: \"kubernetes.io/projected/b1a65958-a086-4f31-b544-8aeebcc054e3-kube-api-access-h9knp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.050050 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.049948    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1a65958-a086-4f31-b544-8aeebcc054e3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv8fg\" (UID: \"b1a65958-a086-4f31-b544-8aeebcc054e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.143299 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.143212    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:53.366748 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:00:53.366722    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a65958_a086_4f31_b544_8aeebcc054e3.slice/crio-4ee7873cb64cd2b76d2b2ee4eae258c0cc2448b8fffc83c02a9459bd92024b04 WatchSource:0}: Error finding container 4ee7873cb64cd2b76d2b2ee4eae258c0cc2448b8fffc83c02a9459bd92024b04: Status 404 returned error can't find the container with id 4ee7873cb64cd2b76d2b2ee4eae258c0cc2448b8fffc83c02a9459bd92024b04
Jul 10 16:00:53.367213 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.367193    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"]
Jul 10 16:00:53.790218 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:53.790183    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg" event={"ID":"b1a65958-a086-4f31-b544-8aeebcc054e3","Type":"ContainerStarted","Data":"4ee7873cb64cd2b76d2b2ee4eae258c0cc2448b8fffc83c02a9459bd92024b04"}
Jul 10 16:00:56.179541 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.179497    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:00:56.179837 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.179586    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:00:56.179837 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.179620    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:00:56.319094 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.319053    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"]
Jul 10 16:00:56.322199 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.322176    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.327662 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.327637    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 16:00:56.329727 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.329707    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 16:00:56.329829 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.329707    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 10 16:00:56.330770 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.330745    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-l6gw6\""
Jul 10 16:00:56.331165 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.331148    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 10 16:00:56.331433 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.331396    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 10 16:00:56.346118 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.346086    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"]
Jul 10 16:00:56.357118 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.357087    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9b44\" (UniqueName: \"kubernetes.io/projected/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-kube-api-access-k9b44\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.357118 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.357122    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-manager-config\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.357324 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.357164    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-metrics-cert\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.357324 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.357251    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-cert\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.457757 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.457723    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9b44\" (UniqueName: \"kubernetes.io/projected/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-kube-api-access-k9b44\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.457757 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.457761    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-manager-config\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.458006 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.457792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-metrics-cert\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.458006 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.457821    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-cert\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.458462 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.458435    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-manager-config\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.460445 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.460398    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-metrics-cert\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.460547 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.460475    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-cert\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.493022 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.492985    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9b44\" (UniqueName: \"kubernetes.io/projected/d3f16d5e-b928-44fc-aa0d-b7cf3779a615-kube-api-access-k9b44\") pod \"lws-controller-manager-66568bbc8f-bpgkk\" (UID: \"d3f16d5e-b928-44fc-aa0d-b7cf3779a615\") " pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.633055 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.633016    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:00:56.786222 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.786192    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"]
Jul 10 16:00:56.789502 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:00:56.789474    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f16d5e_b928_44fc_aa0d_b7cf3779a615.slice/crio-e209c59e184fe2965478547bc5124453fe01b3d9e8255959e161a8538d087272 WatchSource:0}: Error finding container e209c59e184fe2965478547bc5124453fe01b3d9e8255959e161a8538d087272: Status 404 returned error can't find the container with id e209c59e184fe2965478547bc5124453fe01b3d9e8255959e161a8538d087272
Jul 10 16:00:56.805692 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.805660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk" event={"ID":"d3f16d5e-b928-44fc-aa0d-b7cf3779a615","Type":"ContainerStarted","Data":"e209c59e184fe2965478547bc5124453fe01b3d9e8255959e161a8538d087272"}
Jul 10 16:00:56.807259 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:56.807232    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg" event={"ID":"b1a65958-a086-4f31-b544-8aeebcc054e3","Type":"ContainerStarted","Data":"627289c275009a8a62d2d9c7f0c17920cc15ee14ff85023d4baad835b25af27c"}
Jul 10 16:00:57.144303 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:57.144210    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:57.149188 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:57.149158    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:57.191150 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:57.191098    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg" podStartSLOduration=2.380529896 podStartE2EDuration="5.191080099s" totalImagesPullingTime="2.810550203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:00:56.873552483 +0000 UTC m=+344.017086321" watchObservedRunningTime="2026-07-10 16:00:57.191080099 +0000 UTC m=+344.334613937"
Jul 10 16:00:57.812868 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:57.812829    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:57.814364 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:57.814337    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv8fg"
Jul 10 16:00:59.777876 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:59.777845    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-n7b4v"
Jul 10 16:00:59.823906 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:59.823868    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk" event={"ID":"d3f16d5e-b928-44fc-aa0d-b7cf3779a615","Type":"ContainerStarted","Data":"7512d911c14c4716c72253572a7643baae6dad8aec5175b260c010aa76ea4433"}
Jul 10 16:00:59.824106 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:00:59.824057    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:01:01.531389 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.531302    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk" podStartSLOduration=3.248255603 podStartE2EDuration="5.531269659s" totalImagesPullingTime="2.283014056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:00:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:00:59.884560738 +0000 UTC m=+347.028094578" watchObservedRunningTime="2026-07-10 16:01:01.531269659 +0000 UTC m=+348.674803488"
Jul 10 16:01:01.532830 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.532810    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"]
Jul 10 16:01:01.536207 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.536175    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"
Jul 10 16:01:01.554537 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.554508    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 10 16:01:01.555610 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.555592    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-fp4r2\""
Jul 10 16:01:01.575615 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.575579    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"]
Jul 10 16:01:01.608033 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.607992    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5drdk\" (UniqueName: \"kubernetes.io/projected/c2831a0f-80d6-4601-8c09-e4454c84dd25-kube-api-access-5drdk\") pod \"llmisvc-controller-manager-777c6bd7d5-svpd7\" (UID: \"c2831a0f-80d6-4601-8c09-e4454c84dd25\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"
Jul 10 16:01:01.608033 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.608036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2831a0f-80d6-4601-8c09-e4454c84dd25-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-svpd7\" (UID: \"c2831a0f-80d6-4601-8c09-e4454c84dd25\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"
Jul 10 16:01:01.709304 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.709268    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5drdk\" (UniqueName: \"kubernetes.io/projected/c2831a0f-80d6-4601-8c09-e4454c84dd25-kube-api-access-5drdk\") pod \"llmisvc-controller-manager-777c6bd7d5-svpd7\" (UID: \"c2831a0f-80d6-4601-8c09-e4454c84dd25\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"
Jul 10 16:01:01.709543 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.709322    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2831a0f-80d6-4601-8c09-e4454c84dd25-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-svpd7\" (UID: \"c2831a0f-80d6-4601-8c09-e4454c84dd25\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"
Jul 10 16:01:01.712199 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.712167    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2831a0f-80d6-4601-8c09-e4454c84dd25-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-svpd7\" (UID: \"c2831a0f-80d6-4601-8c09-e4454c84dd25\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"
Jul 10 16:01:01.724126 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.724089    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5drdk\" (UniqueName: \"kubernetes.io/projected/c2831a0f-80d6-4601-8c09-e4454c84dd25-kube-api-access-5drdk\") pod \"llmisvc-controller-manager-777c6bd7d5-svpd7\" (UID: \"c2831a0f-80d6-4601-8c09-e4454c84dd25\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"
Jul 10 16:01:01.847244 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:01.847151    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"
Jul 10 16:01:02.020508 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:02.020484    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"]
Jul 10 16:01:02.022608 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:01:02.022583    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc2831a0f_80d6_4601_8c09_e4454c84dd25.slice/crio-d5fad228d75ce28bba546981ce5ed30c73153777861bf73aff06c58f48477c45 WatchSource:0}: Error finding container d5fad228d75ce28bba546981ce5ed30c73153777861bf73aff06c58f48477c45: Status 404 returned error can't find the container with id d5fad228d75ce28bba546981ce5ed30c73153777861bf73aff06c58f48477c45
Jul 10 16:01:02.837583 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:02.837551    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7" event={"ID":"c2831a0f-80d6-4601-8c09-e4454c84dd25","Type":"ContainerStarted","Data":"d5fad228d75ce28bba546981ce5ed30c73153777861bf73aff06c58f48477c45"}
Jul 10 16:01:05.854563 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:05.854475    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7" event={"ID":"c2831a0f-80d6-4601-8c09-e4454c84dd25","Type":"ContainerStarted","Data":"023b10e2783a84c23a8a52e6c906e3906144fda7e8489f491aade9cc073d6c41"}
Jul 10 16:01:05.854914 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:05.854626    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"
Jul 10 16:01:05.881949 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:05.881896    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7" podStartSLOduration=1.450796633 podStartE2EDuration="4.881877066s" totalImagesPullingTime="3.431080433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:01:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:01:05.879430452 +0000 UTC m=+353.022964288" watchObservedRunningTime="2026-07-10 16:01:05.881877066 +0000 UTC m=+353.025410904"
Jul 10 16:01:10.830906 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:10.830870    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66568bbc8f-bpgkk"
Jul 10 16:01:36.867350 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:01:36.867317    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-svpd7"
Jul 10 16:02:25.028362 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:25.028329    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-22bgx"]
Jul 10 16:02:25.030874 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:25.030856    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22bgx"
Jul 10 16:02:25.036545 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:25.036515    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-dvzxg\""
Jul 10 16:02:25.037574 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:25.037552    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 16:02:25.037689 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:25.037672    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 16:02:25.052614 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:25.052579    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-22bgx"]
Jul 10 16:02:25.154046 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:25.154004    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm6j9\" (UniqueName: \"kubernetes.io/projected/a0f53ac0-4ac4-4dd9-b501-c7f9de14cde2-kube-api-access-bm6j9\") pod \"authorino-operator-75cc8d5cb7-22bgx\" (UID: \"a0f53ac0-4ac4-4dd9-b501-c7f9de14cde2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-22bgx"
Jul 10 16:02:25.254702 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:25.254666    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm6j9\" (UniqueName: \"kubernetes.io/projected/a0f53ac0-4ac4-4dd9-b501-c7f9de14cde2-kube-api-access-bm6j9\") pod \"authorino-operator-75cc8d5cb7-22bgx\" (UID: \"a0f53ac0-4ac4-4dd9-b501-c7f9de14cde2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-22bgx"
Jul 10 16:02:25.265936 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:25.265903    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm6j9\" (UniqueName: \"kubernetes.io/projected/a0f53ac0-4ac4-4dd9-b501-c7f9de14cde2-kube-api-access-bm6j9\") pod \"authorino-operator-75cc8d5cb7-22bgx\" (UID: \"a0f53ac0-4ac4-4dd9-b501-c7f9de14cde2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-22bgx"
Jul 10 16:02:25.341359 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:25.341262    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22bgx"
Jul 10 16:02:25.485806 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:25.485769    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-22bgx"]
Jul 10 16:02:25.490162 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:02:25.490121    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f53ac0_4ac4_4dd9_b501_c7f9de14cde2.slice/crio-345be1dac2528b7ac12bafb7ea4be3d23c9b4792e3ab96554222f88b0190a933 WatchSource:0}: Error finding container 345be1dac2528b7ac12bafb7ea4be3d23c9b4792e3ab96554222f88b0190a933: Status 404 returned error can't find the container with id 345be1dac2528b7ac12bafb7ea4be3d23c9b4792e3ab96554222f88b0190a933
Jul 10 16:02:26.195883 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:26.195841    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22bgx" event={"ID":"a0f53ac0-4ac4-4dd9-b501-c7f9de14cde2","Type":"ContainerStarted","Data":"345be1dac2528b7ac12bafb7ea4be3d23c9b4792e3ab96554222f88b0190a933"}
Jul 10 16:02:29.210990 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:29.210956    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22bgx" event={"ID":"a0f53ac0-4ac4-4dd9-b501-c7f9de14cde2","Type":"ContainerStarted","Data":"a39951b4460e2404d85ab7bb6eb8181533c8fd0faeb93bade9ebd600836a46f6"}
Jul 10 16:02:29.211450 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:29.211102    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22bgx"
Jul 10 16:02:29.235347 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:29.235288    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22bgx" podStartSLOduration=2.352012923 podStartE2EDuration="5.235265882s" totalImagesPullingTime="2.883252959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:02:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:02:29.234458977 +0000 UTC m=+436.377992815" watchObservedRunningTime="2026-07-10 16:02:29.235265882 +0000 UTC m=+436.378799720"
Jul 10 16:02:40.217577 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:40.217494    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-22bgx"
Jul 10 16:02:52.454978 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:52.454942    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl"]
Jul 10 16:02:52.463527 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:52.463497    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl"
Jul 10 16:02:52.466482 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:52.466456    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-zxngh\""
Jul 10 16:02:52.473252 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:52.473221    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl"]
Jul 10 16:02:52.602669 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:52.602627    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bglbc\" (UniqueName: \"kubernetes.io/projected/8c490cbd-0f85-4557-9921-b674a2c3e808-kube-api-access-bglbc\") pod \"limitador-operator-controller-manager-859d7b78bd-njgsl\" (UID: \"8c490cbd-0f85-4557-9921-b674a2c3e808\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl"
Jul 10 16:02:52.703618 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:52.703574    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bglbc\" (UniqueName: \"kubernetes.io/projected/8c490cbd-0f85-4557-9921-b674a2c3e808-kube-api-access-bglbc\") pod \"limitador-operator-controller-manager-859d7b78bd-njgsl\" (UID: \"8c490cbd-0f85-4557-9921-b674a2c3e808\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl"
Jul 10 16:02:52.715314 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:52.715242    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bglbc\" (UniqueName: \"kubernetes.io/projected/8c490cbd-0f85-4557-9921-b674a2c3e808-kube-api-access-bglbc\") pod \"limitador-operator-controller-manager-859d7b78bd-njgsl\" (UID: \"8c490cbd-0f85-4557-9921-b674a2c3e808\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl"
Jul 10 16:02:52.775023 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:52.774994    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl"
Jul 10 16:02:52.915083 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:52.915058    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl"]
Jul 10 16:02:52.917464 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:02:52.917431    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c490cbd_0f85_4557_9921_b674a2c3e808.slice/crio-7df013a0365a5d9e6abee1d6c8d86ed667f4ef0c3a43dd5da788ff21bcb9a3b3 WatchSource:0}: Error finding container 7df013a0365a5d9e6abee1d6c8d86ed667f4ef0c3a43dd5da788ff21bcb9a3b3: Status 404 returned error can't find the container with id 7df013a0365a5d9e6abee1d6c8d86ed667f4ef0c3a43dd5da788ff21bcb9a3b3
Jul 10 16:02:53.317033 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:53.316990    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl" event={"ID":"8c490cbd-0f85-4557-9921-b674a2c3e808","Type":"ContainerStarted","Data":"7df013a0365a5d9e6abee1d6c8d86ed667f4ef0c3a43dd5da788ff21bcb9a3b3"}
Jul 10 16:02:56.336546 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:56.336515    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl" event={"ID":"8c490cbd-0f85-4557-9921-b674a2c3e808","Type":"ContainerStarted","Data":"9973b823735a36e806c9d01a0925ccefb771b37a1b2c1f5d34a9c3572ae5ac29"}
Jul 10 16:02:56.336956 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:56.336591    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl"
Jul 10 16:02:56.357934 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:02:56.357886    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl" podStartSLOduration=1.429449985 podStartE2EDuration="4.357867464s" totalImagesPullingTime="2.928417479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:02:56.357188018 +0000 UTC m=+463.500721870" watchObservedRunningTime="2026-07-10 16:02:56.357867464 +0000 UTC m=+463.501401302"
Jul 10 16:03:07.343434 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:07.343385    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-njgsl"
Jul 10 16:03:36.038324 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:36.038290    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-677x5"]
Jul 10 16:03:36.041785 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:36.041759    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-677x5"
Jul 10 16:03:36.044718 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:36.044686    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ztqc9\""
Jul 10 16:03:36.052688 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:36.052650    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-677x5"]
Jul 10 16:03:36.180629 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:36.180587    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcptc\" (UniqueName: \"kubernetes.io/projected/fada0d7f-637d-4039-b881-0fc61d4c7836-kube-api-access-vcptc\") pod \"authorino-58cd955675-677x5\" (UID: \"fada0d7f-637d-4039-b881-0fc61d4c7836\") " pod="kuadrant-system/authorino-58cd955675-677x5"
Jul 10 16:03:36.281215 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:36.281171    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcptc\" (UniqueName: \"kubernetes.io/projected/fada0d7f-637d-4039-b881-0fc61d4c7836-kube-api-access-vcptc\") pod \"authorino-58cd955675-677x5\" (UID: \"fada0d7f-637d-4039-b881-0fc61d4c7836\") " pod="kuadrant-system/authorino-58cd955675-677x5"
Jul 10 16:03:36.290356 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:36.290288    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcptc\" (UniqueName: \"kubernetes.io/projected/fada0d7f-637d-4039-b881-0fc61d4c7836-kube-api-access-vcptc\") pod \"authorino-58cd955675-677x5\" (UID: \"fada0d7f-637d-4039-b881-0fc61d4c7836\") " pod="kuadrant-system/authorino-58cd955675-677x5"
Jul 10 16:03:36.357314 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:36.357265    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-677x5"
Jul 10 16:03:36.496670 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:36.496643    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-677x5"]
Jul 10 16:03:36.499008 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:03:36.498977    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfada0d7f_637d_4039_b881_0fc61d4c7836.slice/crio-63e30dc800c7b83db5a17d15d639e57efa9ce2bf745e27fc65978b96bfc71b67 WatchSource:0}: Error finding container 63e30dc800c7b83db5a17d15d639e57efa9ce2bf745e27fc65978b96bfc71b67: Status 404 returned error can't find the container with id 63e30dc800c7b83db5a17d15d639e57efa9ce2bf745e27fc65978b96bfc71b67
Jul 10 16:03:36.509376 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:36.509341    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-677x5" event={"ID":"fada0d7f-637d-4039-b881-0fc61d4c7836","Type":"ContainerStarted","Data":"63e30dc800c7b83db5a17d15d639e57efa9ce2bf745e27fc65978b96bfc71b67"}
Jul 10 16:03:40.530320 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:40.530280    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-677x5" event={"ID":"fada0d7f-637d-4039-b881-0fc61d4c7836","Type":"ContainerStarted","Data":"cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc"}
Jul 10 16:03:40.546450 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:40.546375    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-677x5" podStartSLOduration=0.981043132 podStartE2EDuration="4.546355848s" totalImagesPullingTime="3.565312716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:03:40.544889694 +0000 UTC m=+507.688423532" watchObservedRunningTime="2026-07-10 16:03:40.546355848 +0000 UTC m=+507.689889688"
Jul 10 16:03:41.632556 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:41.632516    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-677x5"]
Jul 10 16:03:42.539957 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:42.539918    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-677x5" podUID="fada0d7f-637d-4039-b881-0fc61d4c7836" containerName="authorino" containerID="cri-o://cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc" gracePeriod=30
Jul 10 16:03:42.794923 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:42.794853    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-677x5"
Jul 10 16:03:42.942600 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:42.942561    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcptc\" (UniqueName: \"kubernetes.io/projected/fada0d7f-637d-4039-b881-0fc61d4c7836-kube-api-access-vcptc\") pod \"fada0d7f-637d-4039-b881-0fc61d4c7836\" (UID: \"fada0d7f-637d-4039-b881-0fc61d4c7836\") "
Jul 10 16:03:42.945040 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:42.945009    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fada0d7f-637d-4039-b881-0fc61d4c7836-kube-api-access-vcptc" (OuterVolumeSpecName: "kube-api-access-vcptc") pod "fada0d7f-637d-4039-b881-0fc61d4c7836" (UID: "fada0d7f-637d-4039-b881-0fc61d4c7836"). InnerVolumeSpecName "kube-api-access-vcptc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:03:43.043636 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:43.043599    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcptc\" (UniqueName: \"kubernetes.io/projected/fada0d7f-637d-4039-b881-0fc61d4c7836-kube-api-access-vcptc\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:03:43.545246 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:43.545208    2567 generic.go:356] "Generic (PLEG): container finished" podID="fada0d7f-637d-4039-b881-0fc61d4c7836" containerID="cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc" exitCode=0
Jul 10 16:03:43.545452 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:43.545264    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-677x5"
Jul 10 16:03:43.545452 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:43.545269    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-677x5" event={"ID":"fada0d7f-637d-4039-b881-0fc61d4c7836","Type":"ContainerDied","Data":"cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc"}
Jul 10 16:03:43.545452 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:43.545375    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-677x5" event={"ID":"fada0d7f-637d-4039-b881-0fc61d4c7836","Type":"ContainerDied","Data":"63e30dc800c7b83db5a17d15d639e57efa9ce2bf745e27fc65978b96bfc71b67"}
Jul 10 16:03:43.545452 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:43.545392    2567 scope.go:117] "RemoveContainer" containerID="cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc"
Jul 10 16:03:43.553670 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:43.553644    2567 scope.go:117] "RemoveContainer" containerID="cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc"
Jul 10 16:03:43.553986 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:03:43.553969    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc\": container with ID starting with cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc not found: ID does not exist" containerID="cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc"
Jul 10 16:03:43.554043 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:43.553995    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc"} err="failed to get container status \"cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc\": rpc error: code = NotFound desc = could not find container \"cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc\": container with ID starting with cc47fcc4a2e5784d4f68b7559f9a616c16f64655db10b48095ce7443f745bddc not found: ID does not exist"
Jul 10 16:03:43.580866 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:43.580837    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-677x5"]
Jul 10 16:03:43.593277 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:43.593249    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-677x5"]
Jul 10 16:03:45.404913 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:03:45.404873    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fada0d7f-637d-4039-b881-0fc61d4c7836" path="/var/lib/kubelet/pods/fada0d7f-637d-4039-b881-0fc61d4c7836/volumes"
Jul 10 16:04:07.491416 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.491323    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nl7rc"]
Jul 10 16:04:07.491861 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.491715    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fada0d7f-637d-4039-b881-0fc61d4c7836" containerName="authorino"
Jul 10 16:04:07.491861 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.491730    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fada0d7f-637d-4039-b881-0fc61d4c7836" containerName="authorino"
Jul 10 16:04:07.491861 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.491797    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fada0d7f-637d-4039-b881-0fc61d4c7836" containerName="authorino"
Jul 10 16:04:07.522894 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.522855    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nl7rc"]
Jul 10 16:04:07.523056 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.522955    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-nl7rc"
Jul 10 16:04:07.525858 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.525832    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ztqc9\""
Jul 10 16:04:07.568728 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.568687    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pttm\" (UniqueName: \"kubernetes.io/projected/4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8-kube-api-access-8pttm\") pod \"authorino-5d4b67c446-nl7rc\" (UID: \"4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8\") " pod="kuadrant-system/authorino-5d4b67c446-nl7rc"
Jul 10 16:04:07.669583 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.669548    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pttm\" (UniqueName: \"kubernetes.io/projected/4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8-kube-api-access-8pttm\") pod \"authorino-5d4b67c446-nl7rc\" (UID: \"4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8\") " pod="kuadrant-system/authorino-5d4b67c446-nl7rc"
Jul 10 16:04:07.679534 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.679505    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pttm\" (UniqueName: \"kubernetes.io/projected/4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8-kube-api-access-8pttm\") pod \"authorino-5d4b67c446-nl7rc\" (UID: \"4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8\") " pod="kuadrant-system/authorino-5d4b67c446-nl7rc"
Jul 10 16:04:07.721277 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.721239    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nl7rc"]
Jul 10 16:04:07.721522 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.721509    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-nl7rc"
Jul 10 16:04:07.750918 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.750876    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75b67b7989-6t6h5"]
Jul 10 16:04:07.773869 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.773840    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75b67b7989-6t6h5"]
Jul 10 16:04:07.774032 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.773937    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b67b7989-6t6h5"
Jul 10 16:04:07.871126 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.871084    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4crw\" (UniqueName: \"kubernetes.io/projected/23212882-7fd2-487e-83aa-abc087fdc017-kube-api-access-z4crw\") pod \"authorino-75b67b7989-6t6h5\" (UID: \"23212882-7fd2-487e-83aa-abc087fdc017\") " pod="kuadrant-system/authorino-75b67b7989-6t6h5"
Jul 10 16:04:07.883763 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.883305    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nl7rc"]
Jul 10 16:04:07.886208 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:04:07.886181    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b6ac0ef_5c06_4489_9fa3_69df8d2eafb8.slice/crio-d2f80739823a98903bf9884c1e0a3e10c26fb2b8b2efa93d3c8fde7aed393844 WatchSource:0}: Error finding container d2f80739823a98903bf9884c1e0a3e10c26fb2b8b2efa93d3c8fde7aed393844: Status 404 returned error can't find the container with id d2f80739823a98903bf9884c1e0a3e10c26fb2b8b2efa93d3c8fde7aed393844
Jul 10 16:04:07.971777 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.971739    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4crw\" (UniqueName: \"kubernetes.io/projected/23212882-7fd2-487e-83aa-abc087fdc017-kube-api-access-z4crw\") pod \"authorino-75b67b7989-6t6h5\" (UID: \"23212882-7fd2-487e-83aa-abc087fdc017\") " pod="kuadrant-system/authorino-75b67b7989-6t6h5"
Jul 10 16:04:07.981226 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:07.981192    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4crw\" (UniqueName: \"kubernetes.io/projected/23212882-7fd2-487e-83aa-abc087fdc017-kube-api-access-z4crw\") pod \"authorino-75b67b7989-6t6h5\" (UID: \"23212882-7fd2-487e-83aa-abc087fdc017\") " pod="kuadrant-system/authorino-75b67b7989-6t6h5"
Jul 10 16:04:08.026524 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.026435    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75b67b7989-6t6h5"]
Jul 10 16:04:08.026707 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.026696    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b67b7989-6t6h5"
Jul 10 16:04:08.055879 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.055850    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6cf4b7dbf-7h2m6"]
Jul 10 16:04:08.058031 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.058014    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6"
Jul 10 16:04:08.060920 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.060898    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 10 16:04:08.068739 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.068711    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cf4b7dbf-7h2m6"]
Jul 10 16:04:08.173821 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.173774    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wvhc\" (UniqueName: \"kubernetes.io/projected/c8a789f7-ae11-4c17-b8b3-20a588e983e5-kube-api-access-7wvhc\") pod \"authorino-6cf4b7dbf-7h2m6\" (UID: \"c8a789f7-ae11-4c17-b8b3-20a588e983e5\") " pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6"
Jul 10 16:04:08.173821 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.173830    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8a789f7-ae11-4c17-b8b3-20a588e983e5-tls-cert\") pod \"authorino-6cf4b7dbf-7h2m6\" (UID: \"c8a789f7-ae11-4c17-b8b3-20a588e983e5\") " pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6"
Jul 10 16:04:08.177976 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.177951    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75b67b7989-6t6h5"]
Jul 10 16:04:08.179677 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:04:08.179644    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23212882_7fd2_487e_83aa_abc087fdc017.slice/crio-3237ecbb22cd7605b1d565845ef91510108164128b46425411c7a9412cbc71e0 WatchSource:0}: Error finding container 3237ecbb22cd7605b1d565845ef91510108164128b46425411c7a9412cbc71e0: Status 404 returned error can't find the container with id 3237ecbb22cd7605b1d565845ef91510108164128b46425411c7a9412cbc71e0
Jul 10 16:04:08.274880 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.274858    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8a789f7-ae11-4c17-b8b3-20a588e983e5-tls-cert\") pod \"authorino-6cf4b7dbf-7h2m6\" (UID: \"c8a789f7-ae11-4c17-b8b3-20a588e983e5\") " pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6"
Jul 10 16:04:08.274992 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.274976    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wvhc\" (UniqueName: \"kubernetes.io/projected/c8a789f7-ae11-4c17-b8b3-20a588e983e5-kube-api-access-7wvhc\") pod \"authorino-6cf4b7dbf-7h2m6\" (UID: \"c8a789f7-ae11-4c17-b8b3-20a588e983e5\") " pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6"
Jul 10 16:04:08.277675 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.277616    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8a789f7-ae11-4c17-b8b3-20a588e983e5-tls-cert\") pod \"authorino-6cf4b7dbf-7h2m6\" (UID: \"c8a789f7-ae11-4c17-b8b3-20a588e983e5\") " pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6"
Jul 10 16:04:08.283978 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.283954    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wvhc\" (UniqueName: \"kubernetes.io/projected/c8a789f7-ae11-4c17-b8b3-20a588e983e5-kube-api-access-7wvhc\") pod \"authorino-6cf4b7dbf-7h2m6\" (UID: \"c8a789f7-ae11-4c17-b8b3-20a588e983e5\") " pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6"
Jul 10 16:04:08.369249 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.369209    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6"
Jul 10 16:04:08.512995 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.512962    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cf4b7dbf-7h2m6"]
Jul 10 16:04:08.516070 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:04:08.516046    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8a789f7_ae11_4c17_b8b3_20a588e983e5.slice/crio-5c323110bb4ec5757ca2d3fc86976c399de2b1dd49c71313eb303709cb34489c WatchSource:0}: Error finding container 5c323110bb4ec5757ca2d3fc86976c399de2b1dd49c71313eb303709cb34489c: Status 404 returned error can't find the container with id 5c323110bb4ec5757ca2d3fc86976c399de2b1dd49c71313eb303709cb34489c
Jul 10 16:04:08.655586 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.655552    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6" event={"ID":"c8a789f7-ae11-4c17-b8b3-20a588e983e5","Type":"ContainerStarted","Data":"5c323110bb4ec5757ca2d3fc86976c399de2b1dd49c71313eb303709cb34489c"}
Jul 10 16:04:08.657486 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.657445    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-nl7rc" event={"ID":"4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8","Type":"ContainerStarted","Data":"85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f"}
Jul 10 16:04:08.657751 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.657493    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-nl7rc" event={"ID":"4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8","Type":"ContainerStarted","Data":"d2f80739823a98903bf9884c1e0a3e10c26fb2b8b2efa93d3c8fde7aed393844"}
Jul 10 16:04:08.657751 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.657508    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-nl7rc" podUID="4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8" containerName="authorino" containerID="cri-o://85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f" gracePeriod=30
Jul 10 16:04:08.659576 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.659480    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b67b7989-6t6h5" event={"ID":"23212882-7fd2-487e-83aa-abc087fdc017","Type":"ContainerStarted","Data":"a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c"}
Jul 10 16:04:08.659705 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.659589    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b67b7989-6t6h5" event={"ID":"23212882-7fd2-487e-83aa-abc087fdc017","Type":"ContainerStarted","Data":"3237ecbb22cd7605b1d565845ef91510108164128b46425411c7a9412cbc71e0"}
Jul 10 16:04:08.659759 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.659709    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75b67b7989-6t6h5" podUID="23212882-7fd2-487e-83aa-abc087fdc017" containerName="authorino" containerID="cri-o://a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c" gracePeriod=30
Jul 10 16:04:08.676293 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.676240    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-nl7rc" podStartSLOduration=1.29979614 podStartE2EDuration="1.676224681s" totalImagesPullingTime="376.428541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:04:08.673979111 +0000 UTC m=+535.817512952" watchObservedRunningTime="2026-07-10 16:04:08.676224681 +0000 UTC m=+535.819758519"
Jul 10 16:04:08.693778 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.693729    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75b67b7989-6t6h5" podStartSLOduration=1.349838318 podStartE2EDuration="1.69371239s" totalImagesPullingTime="343.874072ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:04:08.693378797 +0000 UTC m=+535.836912637" watchObservedRunningTime="2026-07-10 16:04:08.69371239 +0000 UTC m=+535.837246229"
Jul 10 16:04:08.929095 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.929066    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b67b7989-6t6h5"
Jul 10 16:04:08.943751 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.943719    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-nl7rc"
Jul 10 16:04:08.981050 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.981017    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pttm\" (UniqueName: \"kubernetes.io/projected/4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8-kube-api-access-8pttm\") pod \"4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8\" (UID: \"4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8\") "
Jul 10 16:04:08.981251 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.981081    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4crw\" (UniqueName: \"kubernetes.io/projected/23212882-7fd2-487e-83aa-abc087fdc017-kube-api-access-z4crw\") pod \"23212882-7fd2-487e-83aa-abc087fdc017\" (UID: \"23212882-7fd2-487e-83aa-abc087fdc017\") "
Jul 10 16:04:08.983632 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.983589    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23212882-7fd2-487e-83aa-abc087fdc017-kube-api-access-z4crw" (OuterVolumeSpecName: "kube-api-access-z4crw") pod "23212882-7fd2-487e-83aa-abc087fdc017" (UID: "23212882-7fd2-487e-83aa-abc087fdc017"). InnerVolumeSpecName "kube-api-access-z4crw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:04:08.983799 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:08.983756    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8-kube-api-access-8pttm" (OuterVolumeSpecName: "kube-api-access-8pttm") pod "4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8" (UID: "4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8"). InnerVolumeSpecName "kube-api-access-8pttm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:04:09.082311 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.082280    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4crw\" (UniqueName: \"kubernetes.io/projected/23212882-7fd2-487e-83aa-abc087fdc017-kube-api-access-z4crw\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:04:09.082311 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.082306    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pttm\" (UniqueName: \"kubernetes.io/projected/4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8-kube-api-access-8pttm\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:04:09.665007 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.664924    2567 generic.go:356] "Generic (PLEG): container finished" podID="23212882-7fd2-487e-83aa-abc087fdc017" containerID="a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c" exitCode=0
Jul 10 16:04:09.665007 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.664973    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b67b7989-6t6h5"
Jul 10 16:04:09.665007 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.664973    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b67b7989-6t6h5" event={"ID":"23212882-7fd2-487e-83aa-abc087fdc017","Type":"ContainerDied","Data":"a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c"}
Jul 10 16:04:09.665564 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.665017    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b67b7989-6t6h5" event={"ID":"23212882-7fd2-487e-83aa-abc087fdc017","Type":"ContainerDied","Data":"3237ecbb22cd7605b1d565845ef91510108164128b46425411c7a9412cbc71e0"}
Jul 10 16:04:09.665564 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.665040    2567 scope.go:117] "RemoveContainer" containerID="a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c"
Jul 10 16:04:09.666724 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.666693    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6" event={"ID":"c8a789f7-ae11-4c17-b8b3-20a588e983e5","Type":"ContainerStarted","Data":"9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a"}
Jul 10 16:04:09.668274 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.668248    2567 generic.go:356] "Generic (PLEG): container finished" podID="4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8" containerID="85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f" exitCode=0
Jul 10 16:04:09.668393 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.668285    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-nl7rc"
Jul 10 16:04:09.668393 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.668299    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-nl7rc" event={"ID":"4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8","Type":"ContainerDied","Data":"85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f"}
Jul 10 16:04:09.668393 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.668328    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-nl7rc" event={"ID":"4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8","Type":"ContainerDied","Data":"d2f80739823a98903bf9884c1e0a3e10c26fb2b8b2efa93d3c8fde7aed393844"}
Jul 10 16:04:09.674186 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.674169    2567 scope.go:117] "RemoveContainer" containerID="a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c"
Jul 10 16:04:09.674454 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:04:09.674434    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c\": container with ID starting with a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c not found: ID does not exist" containerID="a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c"
Jul 10 16:04:09.674500 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.674461    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c"} err="failed to get container status \"a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c\": rpc error: code = NotFound desc = could not find container \"a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c\": container with ID starting with a29d90e04f635f2437c94638e20c87c253d25f8d8c3bc9061b35d6b14ea0e40c not found: ID does not exist"
Jul 10 16:04:09.674500 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.674476    2567 scope.go:117] "RemoveContainer" containerID="85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f"
Jul 10 16:04:09.681763 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.681739    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75b67b7989-6t6h5"]
Jul 10 16:04:09.682785 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.682765    2567 scope.go:117] "RemoveContainer" containerID="85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f"
Jul 10 16:04:09.683064 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:04:09.683046    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f\": container with ID starting with 85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f not found: ID does not exist" containerID="85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f"
Jul 10 16:04:09.683114 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.683072    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f"} err="failed to get container status \"85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f\": rpc error: code = NotFound desc = could not find container \"85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f\": container with ID starting with 85a2d9e7ff9c364dcc27bd033cdd3dba795e462c28c7f95ecb8b024e2cfa161f not found: ID does not exist"
Jul 10 16:04:09.686158 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.686139    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75b67b7989-6t6h5"]
Jul 10 16:04:09.702434 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.702371    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6" podStartSLOduration=1.373644395 podStartE2EDuration="1.70235509s" totalImagesPullingTime="328.710695ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:04:09.700615533 +0000 UTC m=+536.844149374" watchObservedRunningTime="2026-07-10 16:04:09.70235509 +0000 UTC m=+536.845888937"
Jul 10 16:04:09.723749 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.723650    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nl7rc"]
Jul 10 16:04:09.729639 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:09.729612    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nl7rc"]
Jul 10 16:04:11.405299 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:11.405265    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23212882-7fd2-487e-83aa-abc087fdc017" path="/var/lib/kubelet/pods/23212882-7fd2-487e-83aa-abc087fdc017/volumes"
Jul 10 16:04:11.405703 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:04:11.405614    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8" path="/var/lib/kubelet/pods/4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8/volumes"
Jul 10 16:05:22.939183 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.939147    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"]
Jul 10 16:05:22.939623 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.939561    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23212882-7fd2-487e-83aa-abc087fdc017" containerName="authorino"
Jul 10 16:05:22.939623 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.939578    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="23212882-7fd2-487e-83aa-abc087fdc017" containerName="authorino"
Jul 10 16:05:22.939623 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.939590    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8" containerName="authorino"
Jul 10 16:05:22.939623 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.939595    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8" containerName="authorino"
Jul 10 16:05:22.939623 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.939645    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b6ac0ef-5c06-4489-9fa3-69df8d2eafb8" containerName="authorino"
Jul 10 16:05:22.939898 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.939660    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="23212882-7fd2-487e-83aa-abc087fdc017" containerName="authorino"
Jul 10 16:05:22.941839 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.941819    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:22.944457 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.944432    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 16:05:22.945646 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.945621    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8j9dd\""
Jul 10 16:05:22.945769 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.945680    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 10 16:05:22.945769 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.945683    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 16:05:22.947766 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.947745    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:22.947887 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.947773    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:22.947887 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.947814    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:22.947887 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.947872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:22.948012 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.947932    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:22.948012 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.947966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdlw8\" (UniqueName: \"kubernetes.io/projected/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-kube-api-access-rdlw8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:22.952966 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:22.952942    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"]
Jul 10 16:05:23.048565 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.048508    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.048752 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.048602    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.048752 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.048647    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.048752 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.048701    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdlw8\" (UniqueName: \"kubernetes.io/projected/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-kube-api-access-rdlw8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.048914 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.048867    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.048914 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.048882    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.049021 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.048923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.049021 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.048959    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.049157 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.049138    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.051295 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.051268    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.051451 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.051318    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.059398 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.059365    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdlw8\" (UniqueName: \"kubernetes.io/projected/6a4dbbf4-0334-4eaf-987a-f01a0a28712d-kube-api-access-rdlw8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5trlm\" (UID: \"6a4dbbf4-0334-4eaf-987a-f01a0a28712d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.253631 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.253597    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:23.396931 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.396908    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"]
Jul 10 16:05:23.399646 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:05:23.399613    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a4dbbf4_0334_4eaf_987a_f01a0a28712d.slice/crio-9a8539ad514e2a7e5475365c38f678badf56fc370d08508519d30ed60683cc7d WatchSource:0}: Error finding container 9a8539ad514e2a7e5475365c38f678badf56fc370d08508519d30ed60683cc7d: Status 404 returned error can't find the container with id 9a8539ad514e2a7e5475365c38f678badf56fc370d08508519d30ed60683cc7d
Jul 10 16:05:23.994458 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:23.994397    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm" event={"ID":"6a4dbbf4-0334-4eaf-987a-f01a0a28712d","Type":"ContainerStarted","Data":"9a8539ad514e2a7e5475365c38f678badf56fc370d08508519d30ed60683cc7d"}
Jul 10 16:05:25.000605 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:25.000565    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm" event={"ID":"6a4dbbf4-0334-4eaf-987a-f01a0a28712d","Type":"ContainerStarted","Data":"0ae4e39f6eba37e33d03c6d8b10153ebcf1b4d0ecbc7e4b60cc1c9d1ceac4b90"}
Jul 10 16:05:25.021042 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:25.020989    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm" podStartSLOduration=1.852075237 podStartE2EDuration="3.020968134s" totalImagesPullingTime="1.168892897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:05:25.020694461 +0000 UTC m=+612.164228304" watchObservedRunningTime="2026-07-10 16:05:25.020968134 +0000 UTC m=+612.164501974"
Jul 10 16:05:33.254475 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:33.254422    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:33.257074 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:33.254501    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:33.267225 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:33.267194    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:05:34.063392 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:05:34.057314    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5trlm"
Jul 10 16:06:00.952111 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:00.952075    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"]
Jul 10 16:06:00.956129 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:00.956108    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:00.958836 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:00.958815    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 10 16:06:00.967319 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:00.967295    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"]
Jul 10 16:06:01.082449 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.082376    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.082625 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.082463    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7234765d-4dc7-4791-82da-2389c8ee16f7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.082625 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.082490    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.082625 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.082611    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.082741 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.082631    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.082741 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.082649    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt2v5\" (UniqueName: \"kubernetes.io/projected/7234765d-4dc7-4791-82da-2389c8ee16f7-kube-api-access-nt2v5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.183826 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.183795    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.184051 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.183831    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.184051 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.183852    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt2v5\" (UniqueName: \"kubernetes.io/projected/7234765d-4dc7-4791-82da-2389c8ee16f7-kube-api-access-nt2v5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.184051 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.183894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.184051 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.183919    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7234765d-4dc7-4791-82da-2389c8ee16f7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.184051 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.183939    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.184305 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.184262    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.184426 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.184373    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.184543 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.184527    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.186433 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.186390    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7234765d-4dc7-4791-82da-2389c8ee16f7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.186558 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.186539    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7234765d-4dc7-4791-82da-2389c8ee16f7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.192809 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.192786    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt2v5\" (UniqueName: \"kubernetes.io/projected/7234765d-4dc7-4791-82da-2389c8ee16f7-kube-api-access-nt2v5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf\" (UID: \"7234765d-4dc7-4791-82da-2389c8ee16f7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.266111 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.266021    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:01.413015 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.412982    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"]
Jul 10 16:06:01.415018 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:06:01.414982    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7234765d_4dc7_4791_82da_2389c8ee16f7.slice/crio-c161ba6654de8c554843d4ab8c2fdd66e31f5f029712b540a1bb2e1e3bc9a99e WatchSource:0}: Error finding container c161ba6654de8c554843d4ab8c2fdd66e31f5f029712b540a1bb2e1e3bc9a99e: Status 404 returned error can't find the container with id c161ba6654de8c554843d4ab8c2fdd66e31f5f029712b540a1bb2e1e3bc9a99e
Jul 10 16:06:01.416501 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:01.416480    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 16:06:02.171604 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:02.171514    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf" event={"ID":"7234765d-4dc7-4791-82da-2389c8ee16f7","Type":"ContainerStarted","Data":"df6df1ca77358aa30d87d54e1e6fcceab4eb5ec65f1dffa40663268c5b56abee"}
Jul 10 16:06:02.171604 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:02.171571    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf" event={"ID":"7234765d-4dc7-4791-82da-2389c8ee16f7","Type":"ContainerStarted","Data":"c161ba6654de8c554843d4ab8c2fdd66e31f5f029712b540a1bb2e1e3bc9a99e"}
Jul 10 16:06:02.196611 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:02.196544    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf" podStartSLOduration=1.911570832 podStartE2EDuration="2.196520741s" totalImagesPullingTime="284.949909ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:06:02.193163389 +0000 UTC m=+649.336697243" watchObservedRunningTime="2026-07-10 16:06:02.196520741 +0000 UTC m=+649.340054585"
Jul 10 16:06:11.266917 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:11.266876    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:11.267471 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:11.266946    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:11.279805 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:11.279776    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:12.231027 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:12.231000    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf"
Jul 10 16:06:34.644232 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.644192    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"]
Jul 10 16:06:34.647540 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.647523    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.650452 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.650428    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 10 16:06:34.658354 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.658330    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"]
Jul 10 16:06:34.785997 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.785940    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qhx2\" (UniqueName: \"kubernetes.io/projected/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-kube-api-access-6qhx2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.786179 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.786006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.786179 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.786048    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.786179 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.786070    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.786179 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.786095    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.786179 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.786119    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.887088 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.887055    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.887088 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.887090    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.887326 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.887111    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.887326 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.887131    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.887326 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.887192    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qhx2\" (UniqueName: \"kubernetes.io/projected/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-kube-api-access-6qhx2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.887326 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.887221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.887641 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.887601    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.887686 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.887633    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.887728 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.887675    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.889515 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.889500    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.889839 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.889823    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.895247 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.895199    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qhx2\" (UniqueName: \"kubernetes.io/projected/cd6528d4-8cfe-4fbd-81b7-419ae29e77a3-kube-api-access-6qhx2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np\" (UID: \"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:34.956581 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:34.956539    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:35.105908 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:35.105853    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"]
Jul 10 16:06:35.108222 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:06:35.108193    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd6528d4_8cfe_4fbd_81b7_419ae29e77a3.slice/crio-a31932a22a1ec12756427955eb42067032f3217c6faa9676f1cb76a96458cf84 WatchSource:0}: Error finding container a31932a22a1ec12756427955eb42067032f3217c6faa9676f1cb76a96458cf84: Status 404 returned error can't find the container with id a31932a22a1ec12756427955eb42067032f3217c6faa9676f1cb76a96458cf84
Jul 10 16:06:35.326556 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:35.326518    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np" event={"ID":"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3","Type":"ContainerStarted","Data":"a31932a22a1ec12756427955eb42067032f3217c6faa9676f1cb76a96458cf84"}
Jul 10 16:06:36.333048 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:36.333014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np" event={"ID":"cd6528d4-8cfe-4fbd-81b7-419ae29e77a3","Type":"ContainerStarted","Data":"fd90bc4b613363b6c1d2c1f4d6edca8c6fefb30ca989bd2fc1a95541cc1c7390"}
Jul 10 16:06:36.353323 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:36.353266    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np" podStartSLOduration=2.099539226 podStartE2EDuration="2.353247084s" totalImagesPullingTime="253.707858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:06:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:06:36.351312313 +0000 UTC m=+683.494846152" watchObservedRunningTime="2026-07-10 16:06:36.353247084 +0000 UTC m=+683.496780923"
Jul 10 16:06:44.957346 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:44.957306    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:44.957856 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:44.957397    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:44.970078 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:44.970050    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:06:45.389574 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:06:45.389476    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np"
Jul 10 16:08:35.329647 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.329556    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6cf4b7dbf-7h2m6"]
Jul 10 16:08:35.330233 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.329738    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6" podUID="c8a789f7-ae11-4c17-b8b3-20a588e983e5" containerName="authorino" containerID="cri-o://9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a" gracePeriod=30
Jul 10 16:08:35.591840 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.591767    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6"
Jul 10 16:08:35.702237 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.702198    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wvhc\" (UniqueName: \"kubernetes.io/projected/c8a789f7-ae11-4c17-b8b3-20a588e983e5-kube-api-access-7wvhc\") pod \"c8a789f7-ae11-4c17-b8b3-20a588e983e5\" (UID: \"c8a789f7-ae11-4c17-b8b3-20a588e983e5\") "
Jul 10 16:08:35.702456 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.702296    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8a789f7-ae11-4c17-b8b3-20a588e983e5-tls-cert\") pod \"c8a789f7-ae11-4c17-b8b3-20a588e983e5\" (UID: \"c8a789f7-ae11-4c17-b8b3-20a588e983e5\") "
Jul 10 16:08:35.704574 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.704543    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8a789f7-ae11-4c17-b8b3-20a588e983e5-kube-api-access-7wvhc" (OuterVolumeSpecName: "kube-api-access-7wvhc") pod "c8a789f7-ae11-4c17-b8b3-20a588e983e5" (UID: "c8a789f7-ae11-4c17-b8b3-20a588e983e5"). InnerVolumeSpecName "kube-api-access-7wvhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:08:35.713528 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.713498    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8a789f7-ae11-4c17-b8b3-20a588e983e5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c8a789f7-ae11-4c17-b8b3-20a588e983e5" (UID: "c8a789f7-ae11-4c17-b8b3-20a588e983e5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 16:08:35.803267 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.803229    2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8a789f7-ae11-4c17-b8b3-20a588e983e5-tls-cert\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:08:35.803267 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.803265    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wvhc\" (UniqueName: \"kubernetes.io/projected/c8a789f7-ae11-4c17-b8b3-20a588e983e5-kube-api-access-7wvhc\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:08:35.877665 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.877585    2567 generic.go:356] "Generic (PLEG): container finished" podID="c8a789f7-ae11-4c17-b8b3-20a588e983e5" containerID="9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a" exitCode=0
Jul 10 16:08:35.877665 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.877634    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6"
Jul 10 16:08:35.877877 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.877670    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6" event={"ID":"c8a789f7-ae11-4c17-b8b3-20a588e983e5","Type":"ContainerDied","Data":"9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a"}
Jul 10 16:08:35.877877 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.877710    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cf4b7dbf-7h2m6" event={"ID":"c8a789f7-ae11-4c17-b8b3-20a588e983e5","Type":"ContainerDied","Data":"5c323110bb4ec5757ca2d3fc86976c399de2b1dd49c71313eb303709cb34489c"}
Jul 10 16:08:35.877877 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.877727    2567 scope.go:117] "RemoveContainer" containerID="9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a"
Jul 10 16:08:35.886666 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.886647    2567 scope.go:117] "RemoveContainer" containerID="9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a"
Jul 10 16:08:35.886964 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:08:35.886946    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a\": container with ID starting with 9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a not found: ID does not exist" containerID="9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a"
Jul 10 16:08:35.887009 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.886974    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a"} err="failed to get container status \"9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a\": rpc error: code = NotFound desc = could not find container \"9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a\": container with ID starting with 9c630bc952d58659bedd28db680ec9527e74367cdba6e170eda7d9cad0fd222a not found: ID does not exist"
Jul 10 16:08:35.911102 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.911066    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6cf4b7dbf-7h2m6"]
Jul 10 16:08:35.915337 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:35.915310    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6cf4b7dbf-7h2m6"]
Jul 10 16:08:37.405004 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:08:37.404970    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8a789f7-ae11-4c17-b8b3-20a588e983e5" path="/var/lib/kubelet/pods/c8a789f7-ae11-4c17-b8b3-20a588e983e5/volumes"
Jul 10 16:22:29.825969 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.825927    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"]
Jul 10 16:22:29.826581 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.826522    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8a789f7-ae11-4c17-b8b3-20a588e983e5" containerName="authorino"
Jul 10 16:22:29.826581 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.826543    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8a789f7-ae11-4c17-b8b3-20a588e983e5" containerName="authorino"
Jul 10 16:22:29.826728 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.826657    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8a789f7-ae11-4c17-b8b3-20a588e983e5" containerName="authorino"
Jul 10 16:22:29.830086 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.830057    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:29.832847 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.832817    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-87e54437-openshift-default-dockercfg-vzztz\""
Jul 10 16:22:29.842918 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.842889    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"]
Jul 10 16:22:29.915915 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.915873    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-socket\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:29.915915 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.915923    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5hk5\" (UniqueName: \"kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-kube-api-access-g5hk5\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:29.916167 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.915947    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-data\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:29.916167 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.915979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-credential-socket\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:29.916167 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.915994    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-certs\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:29.916167 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.916024    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2880f189-980b-43ee-bad5-f37156ff0d40-istiod-ca-cert\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:29.916167 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.916076    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2880f189-980b-43ee-bad5-f37156ff0d40-istio-podinfo\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:29.916167 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.916091    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-istio-token\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:29.916167 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:29.916111    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-envoy\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.016818 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.016780    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-socket\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017020 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.016824    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5hk5\" (UniqueName: \"kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-kube-api-access-g5hk5\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017020 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.016961    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-data\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017142 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.017018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-credential-socket\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017142 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.017043    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-certs\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017142 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.017091    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2880f189-980b-43ee-bad5-f37156ff0d40-istiod-ca-cert\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017287 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.017158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2880f189-980b-43ee-bad5-f37156ff0d40-istio-podinfo\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017287 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.017188    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-socket\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017287 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.017190    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-istio-token\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017287 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.017252    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-envoy\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017517 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.017306    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-data\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017517 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.017400    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-credential-socket\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017608 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.017596    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-certs\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.017866 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.017845    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2880f189-980b-43ee-bad5-f37156ff0d40-istiod-ca-cert\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.019567 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.019544    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-envoy\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.019776 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.019758    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2880f189-980b-43ee-bad5-f37156ff0d40-istio-podinfo\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.025341 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.025317    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5hk5\" (UniqueName: \"kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-kube-api-access-g5hk5\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.025465 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.025359    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-istio-token\") pod \"e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") " pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.144499 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.144385    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:30.311252 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.311224    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"]
Jul 10 16:22:30.313798 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:22:30.313770    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2880f189_980b_43ee_bad5_f37156ff0d40.slice/crio-9083db85a70ad39e5b74d80426b294f3d2c1fe8d9e7cb7c496cbc4ff64b74a9d WatchSource:0}: Error finding container 9083db85a70ad39e5b74d80426b294f3d2c1fe8d9e7cb7c496cbc4ff64b74a9d: Status 404 returned error can't find the container with id 9083db85a70ad39e5b74d80426b294f3d2c1fe8d9e7cb7c496cbc4ff64b74a9d
Jul 10 16:22:30.315899 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.315870    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:22:30.315984 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.315933    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:22:30.315984 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.315961    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:22:30.658846 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.658798    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts" event={"ID":"2880f189-980b-43ee-bad5-f37156ff0d40","Type":"ContainerStarted","Data":"13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e"}
Jul 10 16:22:30.659032 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.658863    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts" event={"ID":"2880f189-980b-43ee-bad5-f37156ff0d40","Type":"ContainerStarted","Data":"9083db85a70ad39e5b74d80426b294f3d2c1fe8d9e7cb7c496cbc4ff64b74a9d"}
Jul 10 16:22:30.682416 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:30.682350    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts" podStartSLOduration=1.682331338 podStartE2EDuration="1.682331338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:22:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:22:30.679080124 +0000 UTC m=+1637.822613965" watchObservedRunningTime="2026-07-10 16:22:30.682331338 +0000 UTC m=+1637.825865177"
Jul 10 16:22:31.144627 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:31.144591    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:31.155342 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:31.155302    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:31.664313 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:31.664282    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:31.665202 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:31.665182    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:42.201935 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:42.201891    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"]
Jul 10 16:22:42.202329 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:42.202199    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts" podUID="2880f189-980b-43ee-bad5-f37156ff0d40" containerName="istio-proxy" containerID="cri-o://13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e" gracePeriod=30
Jul 10 16:22:43.147265 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.147237    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:43.157811 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.157769    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-data\") pod \"2880f189-980b-43ee-bad5-f37156ff0d40\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") "
Jul 10 16:22:43.157811 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.157822    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-istio-token\") pod \"2880f189-980b-43ee-bad5-f37156ff0d40\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") "
Jul 10 16:22:43.158033 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.157843    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-socket\") pod \"2880f189-980b-43ee-bad5-f37156ff0d40\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") "
Jul 10 16:22:43.158033 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.157865    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-certs\") pod \"2880f189-980b-43ee-bad5-f37156ff0d40\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") "
Jul 10 16:22:43.158033 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.157892    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2880f189-980b-43ee-bad5-f37156ff0d40-istio-podinfo\") pod \"2880f189-980b-43ee-bad5-f37156ff0d40\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") "
Jul 10 16:22:43.158033 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.157918    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-credential-socket\") pod \"2880f189-980b-43ee-bad5-f37156ff0d40\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") "
Jul 10 16:22:43.158033 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.157954    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-envoy\") pod \"2880f189-980b-43ee-bad5-f37156ff0d40\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") "
Jul 10 16:22:43.158033 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.157987    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5hk5\" (UniqueName: \"kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-kube-api-access-g5hk5\") pod \"2880f189-980b-43ee-bad5-f37156ff0d40\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") "
Jul 10 16:22:43.158033 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.158035    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2880f189-980b-43ee-bad5-f37156ff0d40-istiod-ca-cert\") pod \"2880f189-980b-43ee-bad5-f37156ff0d40\" (UID: \"2880f189-980b-43ee-bad5-f37156ff0d40\") "
Jul 10 16:22:43.158355 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.158148    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2880f189-980b-43ee-bad5-f37156ff0d40" (UID: "2880f189-980b-43ee-bad5-f37156ff0d40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:22:43.158355 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.158253    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2880f189-980b-43ee-bad5-f37156ff0d40" (UID: "2880f189-980b-43ee-bad5-f37156ff0d40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:22:43.158498 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.158480    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2880f189-980b-43ee-bad5-f37156ff0d40" (UID: "2880f189-980b-43ee-bad5-f37156ff0d40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:22:43.158566 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.158499    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:22:43.158566 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.158523    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-workload-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:22:43.158566 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.158528    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2880f189-980b-43ee-bad5-f37156ff0d40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2880f189-980b-43ee-bad5-f37156ff0d40" (UID: "2880f189-980b-43ee-bad5-f37156ff0d40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 16:22:43.158726 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.158701    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-data" (OuterVolumeSpecName: "istio-data") pod "2880f189-980b-43ee-bad5-f37156ff0d40" (UID: "2880f189-980b-43ee-bad5-f37156ff0d40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:22:43.160462 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.160367    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-istio-token" (OuterVolumeSpecName: "istio-token") pod "2880f189-980b-43ee-bad5-f37156ff0d40" (UID: "2880f189-980b-43ee-bad5-f37156ff0d40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:22:43.160462 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.160387    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-kube-api-access-g5hk5" (OuterVolumeSpecName: "kube-api-access-g5hk5") pod "2880f189-980b-43ee-bad5-f37156ff0d40" (UID: "2880f189-980b-43ee-bad5-f37156ff0d40"). InnerVolumeSpecName "kube-api-access-g5hk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:22:43.160599 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.160467    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2880f189-980b-43ee-bad5-f37156ff0d40" (UID: "2880f189-980b-43ee-bad5-f37156ff0d40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:22:43.160599 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.160493    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2880f189-980b-43ee-bad5-f37156ff0d40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2880f189-980b-43ee-bad5-f37156ff0d40" (UID: "2880f189-980b-43ee-bad5-f37156ff0d40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 16:22:43.258927 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.258880    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-envoy\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:22:43.258927 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.258925    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5hk5\" (UniqueName: \"kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-kube-api-access-g5hk5\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:22:43.258927 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.258939    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2880f189-980b-43ee-bad5-f37156ff0d40-istiod-ca-cert\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:22:43.259379 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.258953    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-istio-data\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:22:43.259379 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.258964    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2880f189-980b-43ee-bad5-f37156ff0d40-istio-token\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:22:43.259379 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.258974    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2880f189-980b-43ee-bad5-f37156ff0d40-istio-podinfo\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:22:43.259379 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.258985    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2880f189-980b-43ee-bad5-f37156ff0d40-credential-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:22:43.726931 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.726894    2567 generic.go:356] "Generic (PLEG): container finished" podID="2880f189-980b-43ee-bad5-f37156ff0d40" containerID="13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e" exitCode=0
Jul 10 16:22:43.727107 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.726943    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"
Jul 10 16:22:43.727107 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.726947    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts" event={"ID":"2880f189-980b-43ee-bad5-f37156ff0d40","Type":"ContainerDied","Data":"13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e"}
Jul 10 16:22:43.727107 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.727004    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts" event={"ID":"2880f189-980b-43ee-bad5-f37156ff0d40","Type":"ContainerDied","Data":"9083db85a70ad39e5b74d80426b294f3d2c1fe8d9e7cb7c496cbc4ff64b74a9d"}
Jul 10 16:22:43.727107 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.727034    2567 scope.go:117] "RemoveContainer" containerID="13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e"
Jul 10 16:22:43.738623 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.738596    2567 scope.go:117] "RemoveContainer" containerID="13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e"
Jul 10 16:22:43.738938 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:22:43.738916    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e\": container with ID starting with 13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e not found: ID does not exist" containerID="13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e"
Jul 10 16:22:43.739013 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.738951    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e"} err="failed to get container status \"13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e\": rpc error: code = NotFound desc = could not find container \"13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e\": container with ID starting with 13f85e56d92474f49e360130fa4d626782683d066cad3ae1f9cb1bc0cdda052e not found: ID does not exist"
Jul 10 16:22:43.746005 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.745979    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"]
Jul 10 16:22:43.750529 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:43.750506    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-87e54437-openshift-default-68884dfdd9-mlsts"]
Jul 10 16:22:45.405889 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:45.405846    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2880f189-980b-43ee-bad5-f37156ff0d40" path="/var/lib/kubelet/pods/2880f189-980b-43ee-bad5-f37156ff0d40/volumes"
Jul 10 16:22:48.484981 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.484943    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"]
Jul 10 16:22:48.485503 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.485486    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2880f189-980b-43ee-bad5-f37156ff0d40" containerName="istio-proxy"
Jul 10 16:22:48.485555 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.485505    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2880f189-980b-43ee-bad5-f37156ff0d40" containerName="istio-proxy"
Jul 10 16:22:48.485600 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.485587    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2880f189-980b-43ee-bad5-f37156ff0d40" containerName="istio-proxy"
Jul 10 16:22:48.488754 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.488728    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.491287 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.491264    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-bd511652-openshift-default-dockercfg-7db44\""
Jul 10 16:22:48.502730 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.502705    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwxj8\" (UniqueName: \"kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-kube-api-access-pwxj8\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.502852 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.502745    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-credential-socket\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.502852 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.502772    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-socket\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.502971 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.502860    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-certs\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.502971 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.502884    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istiod-ca-cert\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.502971 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.502909    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-podinfo\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.502971 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.502933    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-data\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.502971 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.502956    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-token\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.503214 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.502979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-envoy\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.503696 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.503670    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"]
Jul 10 16:22:48.604238 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604203    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-socket\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.604483 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604298    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-certs\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.604483 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604322    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istiod-ca-cert\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.604483 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604362    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-podinfo\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.604483 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604386    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-data\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.604483 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604430    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-token\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.604483 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604464    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-envoy\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.604825 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604571    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwxj8\" (UniqueName: \"kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-kube-api-access-pwxj8\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.604887 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604841    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-socket\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.604887 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-certs\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.604988 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604899    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-credential-socket\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.604988 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.604942    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-data\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.605217 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.605196    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-credential-socket\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.605605 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.605581    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istiod-ca-cert\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.607020 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.606995    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-envoy\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.607106 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.607079    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-podinfo\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.614546 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.614523    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-token\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.614772 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.614746    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwxj8\" (UniqueName: \"kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-kube-api-access-pwxj8\") pod \"e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") " pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.802338 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.802234    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:48.942977 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.942950    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"]
Jul 10 16:22:48.945224 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:22:48.945193    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a5a6f8_22f4_4cc0_9465_53bbb2cd3af6.slice/crio-43cb9fc48f85ddae14536a5994ee30a1d7fa540241738ebf626299d60d6fdce7 WatchSource:0}: Error finding container 43cb9fc48f85ddae14536a5994ee30a1d7fa540241738ebf626299d60d6fdce7: Status 404 returned error can't find the container with id 43cb9fc48f85ddae14536a5994ee30a1d7fa540241738ebf626299d60d6fdce7
Jul 10 16:22:48.947496 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.947461    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:22:48.947564 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.947536    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:22:48.947606 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:48.947572    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:22:49.757173 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:49.757139    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v" event={"ID":"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6","Type":"ContainerStarted","Data":"aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd"}
Jul 10 16:22:49.757588 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:49.757196    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v" event={"ID":"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6","Type":"ContainerStarted","Data":"43cb9fc48f85ddae14536a5994ee30a1d7fa540241738ebf626299d60d6fdce7"}
Jul 10 16:22:49.793976 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:49.793923    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v" podStartSLOduration=1.793904345 podStartE2EDuration="1.793904345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:22:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:22:49.791643673 +0000 UTC m=+1656.935177512" watchObservedRunningTime="2026-07-10 16:22:49.793904345 +0000 UTC m=+1656.937438183"
Jul 10 16:22:49.802802 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:49.802772    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:50.808106 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:50.808074    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:51.766151 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:51.766123    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:51.767056 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:51.767032    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:22:54.256542 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:54.256502    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"]
Jul 10 16:22:54.782321 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:22:54.782244    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v" podUID="f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" containerName="istio-proxy" containerID="cri-o://aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd" gracePeriod=30
Jul 10 16:23:00.534681 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.534654    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:23:00.633547 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.633440    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwxj8\" (UniqueName: \"kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-kube-api-access-pwxj8\") pod \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") "
Jul 10 16:23:00.633547 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.633498    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-envoy\") pod \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") "
Jul 10 16:23:00.633547 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.633525    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-data\") pod \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") "
Jul 10 16:23:00.633547 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.633542    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-token\") pod \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") "
Jul 10 16:23:00.633878 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.633566    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-podinfo\") pod \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") "
Jul 10 16:23:00.633878 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.633590    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-certs\") pod \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") "
Jul 10 16:23:00.633878 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.633608    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istiod-ca-cert\") pod \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") "
Jul 10 16:23:00.633878 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.633637    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-socket\") pod \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") "
Jul 10 16:23:00.633878 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.633682    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-credential-socket\") pod \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\" (UID: \"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6\") "
Jul 10 16:23:00.634102 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.633958    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" (UID: "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:23:00.634102 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.634054    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:23:00.634212 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.634160    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" (UID: "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:23:00.634477 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.634252    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" (UID: "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:23:00.634613 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.634581    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" (UID: "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 16:23:00.634739 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.634713    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-data" (OuterVolumeSpecName: "istio-data") pod "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" (UID: "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:23:00.635987 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.635958    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-kube-api-access-pwxj8" (OuterVolumeSpecName: "kube-api-access-pwxj8") pod "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" (UID: "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6"). InnerVolumeSpecName "kube-api-access-pwxj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:23:00.636110 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.635988    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" (UID: "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:23:00.636110 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.635999    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" (UID: "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:23:00.636397 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.636377    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" (UID: "f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 16:23:00.735304 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.735268    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwxj8\" (UniqueName: \"kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-kube-api-access-pwxj8\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:23:00.735304 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.735303    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-envoy\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:23:00.735543 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.735333    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-data\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:23:00.735543 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.735344    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-token\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:23:00.735543 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.735355    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istio-podinfo\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:23:00.735543 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.735366    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-istiod-ca-cert\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:23:00.735543 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.735377    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-workload-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:23:00.735543 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.735389    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6-credential-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:23:00.816083 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.816037    2567 generic.go:356] "Generic (PLEG): container finished" podID="f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" containerID="aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd" exitCode=0
Jul 10 16:23:00.816278 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.816101    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"
Jul 10 16:23:00.816278 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.816128    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v" event={"ID":"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6","Type":"ContainerDied","Data":"aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd"}
Jul 10 16:23:00.816278 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.816188    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v" event={"ID":"f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6","Type":"ContainerDied","Data":"43cb9fc48f85ddae14536a5994ee30a1d7fa540241738ebf626299d60d6fdce7"}
Jul 10 16:23:00.816278 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.816211    2567 scope.go:117] "RemoveContainer" containerID="aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd"
Jul 10 16:23:00.825350 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.825318    2567 scope.go:117] "RemoveContainer" containerID="aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd"
Jul 10 16:23:00.825641 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:23:00.825622    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd\": container with ID starting with aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd not found: ID does not exist" containerID="aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd"
Jul 10 16:23:00.825710 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.825650    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd"} err="failed to get container status \"aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd\": rpc error: code = NotFound desc = could not find container \"aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd\": container with ID starting with aa2d374ea4a61e01ec4f33541b7732a7efd584a5606d140707e123a899fd9dbd not found: ID does not exist"
Jul 10 16:23:00.838046 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.838018    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"]
Jul 10 16:23:00.842333 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:00.842305    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-bd511652-openshift-default-6fc57bdc67-svd5v"]
Jul 10 16:23:01.405107 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:23:01.405069    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" path="/var/lib/kubelet/pods/f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6/volumes"
Jul 10 16:25:22.409201 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.409165    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"]
Jul 10 16:25:22.409641 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.409554    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" containerName="istio-proxy"
Jul 10 16:25:22.409641 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.409566    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" containerName="istio-proxy"
Jul 10 16:25:22.409641 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.409621    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a5a6f8-22f4-4cc0-9465-53bbb2cd3af6" containerName="istio-proxy"
Jul 10 16:25:22.411367 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.411352    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.414021 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.414005    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-178db329-openshift-default-dockercfg-c89gz\""
Jul 10 16:25:22.426556 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.426530    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"]
Jul 10 16:25:22.523659 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.523624    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-certs\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.523835 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.523671    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-credential-socket\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.523835 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.523783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-token\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.523835 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.523813    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-data\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.523972 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.523840    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-socket\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.523972 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.523880    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-podinfo\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.523972 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.523912    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44f65c28-90b9-4ca1-919d-f06a67bd573b-istiod-ca-cert\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.523972 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.523959    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-envoy\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.524118 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.523992    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsvs7\" (UniqueName: \"kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-kube-api-access-hsvs7\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.624706 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.624670    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-socket\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.624881 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.624715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-podinfo\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.624881 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.624754    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44f65c28-90b9-4ca1-919d-f06a67bd573b-istiod-ca-cert\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.624881 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.624796    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-envoy\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.624881 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.624828    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsvs7\" (UniqueName: \"kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-kube-api-access-hsvs7\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.624881 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.624881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-certs\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.625112 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.624909    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-credential-socket\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.625112 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.624974    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-token\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.625112 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.625001    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-data\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.625247 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.625159    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-socket\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.625298 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.625274    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-certs\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.625356 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.625325    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-data\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.625458 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.625436    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-credential-socket\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.625635 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.625618    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44f65c28-90b9-4ca1-919d-f06a67bd573b-istiod-ca-cert\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.627306 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.627284    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-envoy\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.627545 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.627527    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-podinfo\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.634997 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.634963    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-token\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.635292 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.635272    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsvs7\" (UniqueName: \"kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-kube-api-access-hsvs7\") pod \"e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") " pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.721531 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.721488    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:22.870911 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.870881    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"]
Jul 10 16:25:22.873463 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:25:22.873431    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f65c28_90b9_4ca1_919d_f06a67bd573b.slice/crio-90b8962f21af5fa5a39f53150cf84f8cf2c8b91e5aad9af6c9606e71dad791c3 WatchSource:0}: Error finding container 90b8962f21af5fa5a39f53150cf84f8cf2c8b91e5aad9af6c9606e71dad791c3: Status 404 returned error can't find the container with id 90b8962f21af5fa5a39f53150cf84f8cf2c8b91e5aad9af6c9606e71dad791c3
Jul 10 16:25:22.875636 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.875601    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:25:22.875737 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.875712    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:25:22.875795 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:22.875762    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:25:23.482613 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:23.482574    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" event={"ID":"44f65c28-90b9-4ca1-919d-f06a67bd573b","Type":"ContainerStarted","Data":"7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e"}
Jul 10 16:25:23.483050 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:23.482627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" event={"ID":"44f65c28-90b9-4ca1-919d-f06a67bd573b","Type":"ContainerStarted","Data":"90b8962f21af5fa5a39f53150cf84f8cf2c8b91e5aad9af6c9606e71dad791c3"}
Jul 10 16:25:23.508953 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:23.508895    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" podStartSLOduration=1.508874391 podStartE2EDuration="1.508874391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:25:23.507258059 +0000 UTC m=+1810.650791923" watchObservedRunningTime="2026-07-10 16:25:23.508874391 +0000 UTC m=+1810.652408227"
Jul 10 16:25:23.722424 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:23.722380    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:23.727885 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:23.727858    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:24.488569 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:24.488535    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:24.489514 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:24.489495    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:38.554247 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:38.554214    2567 generic.go:356] "Generic (PLEG): container finished" podID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerID="7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e" exitCode=0
Jul 10 16:25:38.554247 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:38.554255    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" event={"ID":"44f65c28-90b9-4ca1-919d-f06a67bd573b","Type":"ContainerDied","Data":"7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e"}
Jul 10 16:25:38.554775 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:38.554666    2567 scope.go:117] "RemoveContainer" containerID="7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e"
Jul 10 16:25:38.555774 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:38.555742    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:25:38.555876 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:38.555814    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:25:38.555876 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:38.555849    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:25:39.243156 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.243111    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"]
Jul 10 16:25:39.246036 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.246012    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.248825 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.248795    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-178db329\"/\"kube-root-ca.crt\""
Jul 10 16:25:39.248970 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.248841    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-178db329\"/\"openshift-service-ca.crt\""
Jul 10 16:25:39.249740 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.249721    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-178db329\"/\"default-dockercfg-wpjnw\""
Jul 10 16:25:39.249939 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.249737    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-178db329\"/\"e2e-lifecycle-model-178db329-kserve-self-signed-certs\""
Jul 10 16:25:39.258559 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.258525    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"]
Jul 10 16:25:39.268273 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.268238    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhvct\" (UniqueName: \"kubernetes.io/projected/dff613e6-e781-4cac-8b69-6162f549fc76-kube-api-access-xhvct\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.268453 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.268278    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-tmp-dir\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.268453 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.268306    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-home\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.268453 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.268325    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-model-cache\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.268453 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.268339    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dff613e6-e781-4cac-8b69-6162f549fc76-tls-certs\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.268453 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.268417    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-dshm\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.368945 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.368902    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhvct\" (UniqueName: \"kubernetes.io/projected/dff613e6-e781-4cac-8b69-6162f549fc76-kube-api-access-xhvct\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.369155 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.368960    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-tmp-dir\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.369155 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.369017    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-home\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.369155 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.369052    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-model-cache\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.369155 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.369077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dff613e6-e781-4cac-8b69-6162f549fc76-tls-certs\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.369155 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.369141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-dshm\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.369719 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.369686    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-home\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.369794 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.369759    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-model-cache\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.370010 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.369986    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-tmp-dir\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.373236 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.373198    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-dshm\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.373602 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.373583    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dff613e6-e781-4cac-8b69-6162f549fc76-tls-certs\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.380766 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.380702    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhvct\" (UniqueName: \"kubernetes.io/projected/dff613e6-e781-4cac-8b69-6162f549fc76-kube-api-access-xhvct\") pod \"e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") " pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.488573 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.488519    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:39.557772 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.557678    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:39.560984 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.560959    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" event={"ID":"44f65c28-90b9-4ca1-919d-f06a67bd573b","Type":"ContainerStarted","Data":"4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6"}
Jul 10 16:25:39.721997 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.721910    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:39.722800 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.722767    2567 patch_prober.go:28] interesting pod/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 10 16:25:39.722936 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.722830    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 10 16:25:39.757035 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:39.757005    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"]
Jul 10 16:25:39.759348 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:25:39.759318    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddff613e6_e781_4cac_8b69_6162f549fc76.slice/crio-c5fe1eaf7dbc6168caeff8cbc08e4fbfe9af94afbd2649c65d61e19bf5f7f432 WatchSource:0}: Error finding container c5fe1eaf7dbc6168caeff8cbc08e4fbfe9af94afbd2649c65d61e19bf5f7f432: Status 404 returned error can't find the container with id c5fe1eaf7dbc6168caeff8cbc08e4fbfe9af94afbd2649c65d61e19bf5f7f432
Jul 10 16:25:40.567726 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:40.567683    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m" event={"ID":"dff613e6-e781-4cac-8b69-6162f549fc76","Type":"ContainerStarted","Data":"fdb57268f4ed8b6db5eca29fc468dc52ce8ec01c38c5677dddc2ceeaa4abf9da"}
Jul 10 16:25:40.567726 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:40.567730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m" event={"ID":"dff613e6-e781-4cac-8b69-6162f549fc76","Type":"ContainerStarted","Data":"c5fe1eaf7dbc6168caeff8cbc08e4fbfe9af94afbd2649c65d61e19bf5f7f432"}
Jul 10 16:25:40.592064 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:40.589738    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m" podStartSLOduration=1.5897158999999998 podStartE2EDuration="1.5897159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:25:40.589039588 +0000 UTC m=+1827.732573456" watchObservedRunningTime="2026-07-10 16:25:40.5897159 +0000 UTC m=+1827.733249740"
Jul 10 16:25:40.722120 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:40.722086    2567 patch_prober.go:28] interesting pod/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 10 16:25:40.722287 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:40.722157    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 10 16:25:41.722484 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:41.722447    2567 patch_prober.go:28] interesting pod/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 10 16:25:41.722830 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:41.722510    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 10 16:25:42.722688 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:42.722653    2567 patch_prober.go:28] interesting pod/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 10 16:25:42.723068 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:42.722711    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 10 16:25:43.727731 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:43.727702    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:43.728221 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:43.728096    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:43.728857 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:43.728837    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:25:49.558781 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:49.558747    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:49.558781 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:49.558793    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:49.571678 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:49.571645    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:25:49.626988 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:25:49.626956    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:26:13.586131 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:13.586098    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"]
Jul 10 16:26:28.606913 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.606835    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m" podUID="dff613e6-e781-4cac-8b69-6162f549fc76" containerName="main" containerID="cri-o://fdb57268f4ed8b6db5eca29fc468dc52ce8ec01c38c5677dddc2ceeaa4abf9da" gracePeriod=45
Jul 10 16:26:28.800057 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.800007    2567 generic.go:356] "Generic (PLEG): container finished" podID="dff613e6-e781-4cac-8b69-6162f549fc76" containerID="fdb57268f4ed8b6db5eca29fc468dc52ce8ec01c38c5677dddc2ceeaa4abf9da" exitCode=0
Jul 10 16:26:28.800232 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.800146    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m" event={"ID":"dff613e6-e781-4cac-8b69-6162f549fc76","Type":"ContainerDied","Data":"fdb57268f4ed8b6db5eca29fc468dc52ce8ec01c38c5677dddc2ceeaa4abf9da"}
Jul 10 16:26:28.862679 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.862609    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:26:28.930659 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.930630    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dff613e6-e781-4cac-8b69-6162f549fc76-tls-certs\") pod \"dff613e6-e781-4cac-8b69-6162f549fc76\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") "
Jul 10 16:26:28.930824 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.930672    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhvct\" (UniqueName: \"kubernetes.io/projected/dff613e6-e781-4cac-8b69-6162f549fc76-kube-api-access-xhvct\") pod \"dff613e6-e781-4cac-8b69-6162f549fc76\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") "
Jul 10 16:26:28.930824 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.930702    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-home\") pod \"dff613e6-e781-4cac-8b69-6162f549fc76\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") "
Jul 10 16:26:28.930824 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.930726    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-model-cache\") pod \"dff613e6-e781-4cac-8b69-6162f549fc76\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") "
Jul 10 16:26:28.930824 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.930762    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-tmp-dir\") pod \"dff613e6-e781-4cac-8b69-6162f549fc76\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") "
Jul 10 16:26:28.930824 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.930796    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-dshm\") pod \"dff613e6-e781-4cac-8b69-6162f549fc76\" (UID: \"dff613e6-e781-4cac-8b69-6162f549fc76\") "
Jul 10 16:26:28.931202 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.931010    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-home" (OuterVolumeSpecName: "home") pod "dff613e6-e781-4cac-8b69-6162f549fc76" (UID: "dff613e6-e781-4cac-8b69-6162f549fc76"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:26:28.931202 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.931020    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-model-cache" (OuterVolumeSpecName: "model-cache") pod "dff613e6-e781-4cac-8b69-6162f549fc76" (UID: "dff613e6-e781-4cac-8b69-6162f549fc76"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:26:28.931202 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.931162    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-home\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:26:28.931202 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.931178    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-model-cache\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:26:28.931388 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.931292    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dff613e6-e781-4cac-8b69-6162f549fc76" (UID: "dff613e6-e781-4cac-8b69-6162f549fc76"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:26:28.933332 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.933306    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dff613e6-e781-4cac-8b69-6162f549fc76-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dff613e6-e781-4cac-8b69-6162f549fc76" (UID: "dff613e6-e781-4cac-8b69-6162f549fc76"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 16:26:28.933554 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.933527    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-dshm" (OuterVolumeSpecName: "dshm") pod "dff613e6-e781-4cac-8b69-6162f549fc76" (UID: "dff613e6-e781-4cac-8b69-6162f549fc76"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:26:28.933646 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:28.933623    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dff613e6-e781-4cac-8b69-6162f549fc76-kube-api-access-xhvct" (OuterVolumeSpecName: "kube-api-access-xhvct") pod "dff613e6-e781-4cac-8b69-6162f549fc76" (UID: "dff613e6-e781-4cac-8b69-6162f549fc76"). InnerVolumeSpecName "kube-api-access-xhvct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:26:29.031870 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:29.031827    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-tmp-dir\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:26:29.031870 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:29.031863    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dff613e6-e781-4cac-8b69-6162f549fc76-dshm\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:26:29.031870 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:29.031871    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dff613e6-e781-4cac-8b69-6162f549fc76-tls-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:26:29.031870 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:29.031880    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhvct\" (UniqueName: \"kubernetes.io/projected/dff613e6-e781-4cac-8b69-6162f549fc76-kube-api-access-xhvct\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:26:29.806392 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:29.806360    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"
Jul 10 16:26:29.806881 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:29.806358    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m" event={"ID":"dff613e6-e781-4cac-8b69-6162f549fc76","Type":"ContainerDied","Data":"c5fe1eaf7dbc6168caeff8cbc08e4fbfe9af94afbd2649c65d61e19bf5f7f432"}
Jul 10 16:26:29.806881 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:29.806493    2567 scope.go:117] "RemoveContainer" containerID="fdb57268f4ed8b6db5eca29fc468dc52ce8ec01c38c5677dddc2ceeaa4abf9da"
Jul 10 16:26:29.824954 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:29.824927    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"]
Jul 10 16:26:29.843851 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:29.843809    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-178db329/e2e-lifecycle-model-178db329-kserve-6b8588bcdc-nzm6m"]
Jul 10 16:26:31.404999 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:26:31.404952    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dff613e6-e781-4cac-8b69-6162f549fc76" path="/var/lib/kubelet/pods/dff613e6-e781-4cac-8b69-6162f549fc76/volumes"
Jul 10 16:27:06.371641 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:06.371596    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"]
Jul 10 16:27:06.372166 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:06.371839    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerName="istio-proxy" containerID="cri-o://4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6" gracePeriod=30
Jul 10 16:27:11.636457 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.636378    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:27:11.723991 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.723952    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsvs7\" (UniqueName: \"kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-kube-api-access-hsvs7\") pod \"44f65c28-90b9-4ca1-919d-f06a67bd573b\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") "
Jul 10 16:27:11.724175 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724080    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-data\") pod \"44f65c28-90b9-4ca1-919d-f06a67bd573b\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") "
Jul 10 16:27:11.724175 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724119    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-envoy\") pod \"44f65c28-90b9-4ca1-919d-f06a67bd573b\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") "
Jul 10 16:27:11.724175 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724145    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-credential-socket\") pod \"44f65c28-90b9-4ca1-919d-f06a67bd573b\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") "
Jul 10 16:27:11.724316 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724175    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-token\") pod \"44f65c28-90b9-4ca1-919d-f06a67bd573b\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") "
Jul 10 16:27:11.724316 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724200    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-socket\") pod \"44f65c28-90b9-4ca1-919d-f06a67bd573b\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") "
Jul 10 16:27:11.724316 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724222    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44f65c28-90b9-4ca1-919d-f06a67bd573b-istiod-ca-cert\") pod \"44f65c28-90b9-4ca1-919d-f06a67bd573b\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") "
Jul 10 16:27:11.724316 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724252    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-podinfo\") pod \"44f65c28-90b9-4ca1-919d-f06a67bd573b\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") "
Jul 10 16:27:11.724316 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724283    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-certs\") pod \"44f65c28-90b9-4ca1-919d-f06a67bd573b\" (UID: \"44f65c28-90b9-4ca1-919d-f06a67bd573b\") "
Jul 10 16:27:11.725041 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724640    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "44f65c28-90b9-4ca1-919d-f06a67bd573b" (UID: "44f65c28-90b9-4ca1-919d-f06a67bd573b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:27:11.725041 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724740    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44f65c28-90b9-4ca1-919d-f06a67bd573b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "44f65c28-90b9-4ca1-919d-f06a67bd573b" (UID: "44f65c28-90b9-4ca1-919d-f06a67bd573b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 16:27:11.725041 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724989    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-data" (OuterVolumeSpecName: "istio-data") pod "44f65c28-90b9-4ca1-919d-f06a67bd573b" (UID: "44f65c28-90b9-4ca1-919d-f06a67bd573b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:27:11.725425 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.725006    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "44f65c28-90b9-4ca1-919d-f06a67bd573b" (UID: "44f65c28-90b9-4ca1-919d-f06a67bd573b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:27:11.726449 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.724972    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "44f65c28-90b9-4ca1-919d-f06a67bd573b" (UID: "44f65c28-90b9-4ca1-919d-f06a67bd573b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:27:11.726798 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.726765    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-kube-api-access-hsvs7" (OuterVolumeSpecName: "kube-api-access-hsvs7") pod "44f65c28-90b9-4ca1-919d-f06a67bd573b" (UID: "44f65c28-90b9-4ca1-919d-f06a67bd573b"). InnerVolumeSpecName "kube-api-access-hsvs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:27:11.727705 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.727675    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "44f65c28-90b9-4ca1-919d-f06a67bd573b" (UID: "44f65c28-90b9-4ca1-919d-f06a67bd573b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 16:27:11.733422 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.733330    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "44f65c28-90b9-4ca1-919d-f06a67bd573b" (UID: "44f65c28-90b9-4ca1-919d-f06a67bd573b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:27:11.733756 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.733725    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-token" (OuterVolumeSpecName: "istio-token") pod "44f65c28-90b9-4ca1-919d-f06a67bd573b" (UID: "44f65c28-90b9-4ca1-919d-f06a67bd573b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:27:11.825909 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.825870    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsvs7\" (UniqueName: \"kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-kube-api-access-hsvs7\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:27:11.825909 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.825903    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-data\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:27:11.825909 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.825915    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-envoy\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:27:11.826140 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.825923    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-credential-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:27:11.826140 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.825931    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-token\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:27:11.826140 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.825940    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:27:11.826140 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.825947    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44f65c28-90b9-4ca1-919d-f06a67bd573b-istiod-ca-cert\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:27:11.826140 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.825955    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44f65c28-90b9-4ca1-919d-f06a67bd573b-istio-podinfo\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:27:11.826140 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:11.825963    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44f65c28-90b9-4ca1-919d-f06a67bd573b-workload-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:27:12.001175 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.001140    2567 generic.go:356] "Generic (PLEG): container finished" podID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerID="4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6" exitCode=0
Jul 10 16:27:12.001347 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.001222    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"
Jul 10 16:27:12.001347 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.001220    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" event={"ID":"44f65c28-90b9-4ca1-919d-f06a67bd573b","Type":"ContainerDied","Data":"4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6"}
Jul 10 16:27:12.001455 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.001372    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm" event={"ID":"44f65c28-90b9-4ca1-919d-f06a67bd573b","Type":"ContainerDied","Data":"90b8962f21af5fa5a39f53150cf84f8cf2c8b91e5aad9af6c9606e71dad791c3"}
Jul 10 16:27:12.001455 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.001396    2567 scope.go:117] "RemoveContainer" containerID="4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6"
Jul 10 16:27:12.010761 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.010741    2567 scope.go:117] "RemoveContainer" containerID="7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e"
Jul 10 16:27:12.018879 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.018849    2567 scope.go:117] "RemoveContainer" containerID="4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6"
Jul 10 16:27:12.019145 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:27:12.019123    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6\": container with ID starting with 4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6 not found: ID does not exist" containerID="4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6"
Jul 10 16:27:12.019192 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.019155    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6"} err="failed to get container status \"4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6\": rpc error: code = NotFound desc = could not find container \"4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6\": container with ID starting with 4b07f055d1349e54109aeff4285127d2d54c45200ce87305961a1b04e7f32fb6 not found: ID does not exist"
Jul 10 16:27:12.019192 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.019173    2567 scope.go:117] "RemoveContainer" containerID="7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e"
Jul 10 16:27:12.019386 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:27:12.019368    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e\": container with ID starting with 7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e not found: ID does not exist" containerID="7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e"
Jul 10 16:27:12.019450 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.019392    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e"} err="failed to get container status \"7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e\": rpc error: code = NotFound desc = could not find container \"7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e\": container with ID starting with 7bcbd326fb860e35a1f94f14715879ceae48ec3cf5e4c478c133100ca82ac65e not found: ID does not exist"
Jul 10 16:27:12.024025 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.024000    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"]
Jul 10 16:27:12.029450 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:12.029425    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-178db329-openshift-default-54ffb485b7-5xbhm"]
Jul 10 16:27:13.404863 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:27:13.404829    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" path="/var/lib/kubelet/pods/44f65c28-90b9-4ca1-919d-f06a67bd573b/volumes"
Jul 10 16:28:09.308804 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.308696    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"]
Jul 10 16:28:09.309268 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.309170    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerName="istio-proxy"
Jul 10 16:28:09.309268 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.309183    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerName="istio-proxy"
Jul 10 16:28:09.309268 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.309216    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dff613e6-e781-4cac-8b69-6162f549fc76" containerName="main"
Jul 10 16:28:09.309268 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.309225    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dff613e6-e781-4cac-8b69-6162f549fc76" containerName="main"
Jul 10 16:28:09.309430 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.309308    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dff613e6-e781-4cac-8b69-6162f549fc76" containerName="main"
Jul 10 16:28:09.309430 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.309324    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerName="istio-proxy"
Jul 10 16:28:09.309430 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.309332    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerName="istio-proxy"
Jul 10 16:28:09.313156 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.313136    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.315840 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.315817    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-efe9d3-openshift-default-dockercfg-nmksf\""
Jul 10 16:28:09.323583 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.323558    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"]
Jul 10 16:28:09.442760 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.442721    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-credential-socket\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.442946 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.442774    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-certs\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.442946 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.442797    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-socket\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.442946 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.442813    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-data\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.442946 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.442837    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-envoy\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.442946 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.442855    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjxtf\" (UniqueName: \"kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-kube-api-access-mjxtf\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.443132 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.443026    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ceda867-185d-4c19-ab62-4eca78a60014-istio-podinfo\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.443132 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.443094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ceda867-185d-4c19-ab62-4eca78a60014-istiod-ca-cert\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.443132 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.443122    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-istio-token\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544232 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544187    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-credential-socket\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544400 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-certs\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544400 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544366    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-socket\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544528 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544425    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-data\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544528 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544473    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-envoy\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544528 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544492    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjxtf\" (UniqueName: \"kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-kube-api-access-mjxtf\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544671 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544551    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ceda867-185d-4c19-ab62-4eca78a60014-istio-podinfo\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544671 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544593    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ceda867-185d-4c19-ab62-4eca78a60014-istiod-ca-cert\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544671 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-istio-token\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544671 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544645    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-credential-socket\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544875 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544716    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-certs\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544875 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544854    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-socket\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.544981 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.544931    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-data\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.545370 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.545345    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ceda867-185d-4c19-ab62-4eca78a60014-istiod-ca-cert\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.546960 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.546942    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-envoy\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.547293 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.547272    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ceda867-185d-4c19-ab62-4eca78a60014-istio-podinfo\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.556540 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.556507    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-istio-token\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.556863 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.556846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjxtf\" (UniqueName: \"kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-kube-api-access-mjxtf\") pod \"e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") " pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.624894 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.624804    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:09.774607 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.774570    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"]
Jul 10 16:28:09.776727 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:28:09.776698    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ceda867_185d_4c19_ab62_4eca78a60014.slice/crio-130002887f0c8968640b92b43bacd04ac27d16693eeabadf732ea1a5a5ffbf6e WatchSource:0}: Error finding container 130002887f0c8968640b92b43bacd04ac27d16693eeabadf732ea1a5a5ffbf6e: Status 404 returned error can't find the container with id 130002887f0c8968640b92b43bacd04ac27d16693eeabadf732ea1a5a5ffbf6e
Jul 10 16:28:09.779161 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.779129    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:28:09.779247 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.779192    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:28:09.779247 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:09.779226    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:28:10.272815 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:10.272778    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" event={"ID":"7ceda867-185d-4c19-ab62-4eca78a60014","Type":"ContainerStarted","Data":"036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f"}
Jul 10 16:28:10.273032 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:10.272829    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" event={"ID":"7ceda867-185d-4c19-ab62-4eca78a60014","Type":"ContainerStarted","Data":"130002887f0c8968640b92b43bacd04ac27d16693eeabadf732ea1a5a5ffbf6e"}
Jul 10 16:28:10.296555 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:10.296503    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" podStartSLOduration=1.296484194 podStartE2EDuration="1.296484194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:28:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:28:10.295276303 +0000 UTC m=+1977.438810141" watchObservedRunningTime="2026-07-10 16:28:10.296484194 +0000 UTC m=+1977.440018033"
Jul 10 16:28:10.625431 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:10.625322    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:10.630055 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:10.630027    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:11.286630 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:11.286581    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:11.287667 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:11.287642    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:18.836644 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.836603    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"]
Jul 10 16:28:18.837030 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.836985    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerName="istio-proxy"
Jul 10 16:28:18.837030 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.836997    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f65c28-90b9-4ca1-919d-f06a67bd573b" containerName="istio-proxy"
Jul 10 16:28:18.839745 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.839727    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:18.842120 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.842095    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-0a40f1\"/\"openshift-service-ca.crt\""
Jul 10 16:28:18.842254 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.842127    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-0a40f1\"/\"default-dockercfg-f7gpl\""
Jul 10 16:28:18.843209 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.843181    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-0a40f1\"/\"kube-root-ca.crt\""
Jul 10 16:28:18.843336 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.843315    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-0a40f1\"/\"test-model-0a40f1-kserve-self-signed-certs\""
Jul 10 16:28:18.848521 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.848496    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"]
Jul 10 16:28:18.935747 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.935707    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-dshm\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:18.935747 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.935746    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbgqb\" (UniqueName: \"kubernetes.io/projected/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-kube-api-access-dbgqb\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:18.936017 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.935774    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tls-certs\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:18.936017 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.935907    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-home\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:18.936017 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.935959    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-model-cache\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:18.936170 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:18.936051    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tmp-dir\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.037487 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.037441    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tmp-dir\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.037696 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.037503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-dshm\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.037696 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.037521    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbgqb\" (UniqueName: \"kubernetes.io/projected/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-kube-api-access-dbgqb\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.037696 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.037544    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tls-certs\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.037696 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.037585    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-home\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.037696 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.037624    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-model-cache\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.037976 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.037877    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tmp-dir\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.037976 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.037951    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-home\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.038077 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.038045    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-model-cache\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.040054 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.040020    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-dshm\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.040335 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.040317    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tls-certs\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.045644 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.045616    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbgqb\" (UniqueName: \"kubernetes.io/projected/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-kube-api-access-dbgqb\") pod \"test-model-0a40f1-kserve-6cbbb957c7-rfkss\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") " pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.150089 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.149998    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:19.295677 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.295649    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"]
Jul 10 16:28:19.299343 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:28:19.299300    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ce5b52d_9cfb_434c_9ad6_5ee91b3b2c85.slice/crio-33cc87072afd344b12cf8a16438f27b01591cf1927e6248e9b01406b012b78eb WatchSource:0}: Error finding container 33cc87072afd344b12cf8a16438f27b01591cf1927e6248e9b01406b012b78eb: Status 404 returned error can't find the container with id 33cc87072afd344b12cf8a16438f27b01591cf1927e6248e9b01406b012b78eb
Jul 10 16:28:19.323093 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:19.323064    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss" event={"ID":"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85","Type":"ContainerStarted","Data":"33cc87072afd344b12cf8a16438f27b01591cf1927e6248e9b01406b012b78eb"}
Jul 10 16:28:20.328833 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:20.328790    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss" event={"ID":"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85","Type":"ContainerStarted","Data":"5b85916572aac015704331a367846b23705874cf2410c1d35d319597877182ec"}
Jul 10 16:28:20.347551 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:20.347484    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss" podStartSLOduration=2.347457173 podStartE2EDuration="2.347457173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:28:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:28:20.344834554 +0000 UTC m=+1987.488368409" watchObservedRunningTime="2026-07-10 16:28:20.347457173 +0000 UTC m=+1987.490991015"
Jul 10 16:28:27.328606 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:27.328560    2567 patch_prober.go:28] interesting pod/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 16:28:27.329003 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:27.328629    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 16:28:28.369069 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:28.369035    2567 generic.go:356] "Generic (PLEG): container finished" podID="7ceda867-185d-4c19-ab62-4eca78a60014" containerID="036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f" exitCode=0
Jul 10 16:28:28.369543 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:28.369105    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" event={"ID":"7ceda867-185d-4c19-ab62-4eca78a60014","Type":"ContainerDied","Data":"036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f"}
Jul 10 16:28:28.369543 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:28.369512    2567 scope.go:117] "RemoveContainer" containerID="036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f"
Jul 10 16:28:28.370526 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:28.370499    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:28:28.370609 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:28.370570    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:28:28.370609 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:28.370604    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:28:29.150509 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:29.150477    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:29.150509 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:29.150526    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:29.163826 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:29.163798    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:29.375578 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:29.375547    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" event={"ID":"7ceda867-185d-4c19-ab62-4eca78a60014","Type":"ContainerStarted","Data":"323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c"}
Jul 10 16:28:29.413352 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:29.413269    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:28:29.625792 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:29.625750    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:29.626067 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:29.626041    2567 patch_prober.go:28] interesting pod/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 10 16:28:29.626119 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:29.626096    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 10 16:28:30.625767 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:30.625727    2567 patch_prober.go:28] interesting pod/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 10 16:28:30.626164 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:30.625785    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 10 16:28:31.626207 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:31.626168    2567 patch_prober.go:28] interesting pod/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 10 16:28:31.626647 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:31.626228    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 10 16:28:32.625236 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:32.625198    2567 patch_prober.go:28] interesting pod/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 10 16:28:32.625471 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:32.625254    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 10 16:28:33.628894 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:33.628856    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:33.629321 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:33.629198    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:28:33.630065 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:28:33.630036    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:29:02.020635 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.020595    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"]
Jul 10 16:29:02.024328 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.024304    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.027999 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.027969    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-efe9d3\"/\"default-dockercfg-lrn9b\""
Jul 10 16:29:02.028165 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.028030    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-efe9d3\"/\"openshift-service-ca.crt\""
Jul 10 16:29:02.028165 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.028073    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-efe9d3\"/\"kube-root-ca.crt\""
Jul 10 16:29:02.028165 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.028138    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-efe9d3\"/\"test-model-efe9d3-kserve-self-signed-certs\""
Jul 10 16:29:02.034201 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.034174    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"]
Jul 10 16:29:02.140774 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.140730    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76b737af-0729-447d-babe-5328963939e2-tls-certs\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.140981 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.140792    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-dshm\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.140981 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.140810    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-tmp-dir\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.140981 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.140862    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl62n\" (UniqueName: \"kubernetes.io/projected/76b737af-0729-447d-babe-5328963939e2-kube-api-access-xl62n\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.140981 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.140899    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-home\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.140981 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.140962    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-model-cache\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.241877 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.241835    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76b737af-0729-447d-babe-5328963939e2-tls-certs\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.242084 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.241903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-dshm\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.242084 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.241923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-tmp-dir\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.242084 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.242023    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl62n\" (UniqueName: \"kubernetes.io/projected/76b737af-0729-447d-babe-5328963939e2-kube-api-access-xl62n\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.242084 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.242063    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-home\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.242297 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.242100    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-model-cache\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.242297 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.242274    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-tmp-dir\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.242496 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.242477    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-model-cache\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.242548 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.242493    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-home\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.244591 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.244566    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-dshm\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.244766 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.244750    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76b737af-0729-447d-babe-5328963939e2-tls-certs\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.250423 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.250383    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl62n\" (UniqueName: \"kubernetes.io/projected/76b737af-0729-447d-babe-5328963939e2-kube-api-access-xl62n\") pod \"test-model-efe9d3-kserve-d896f86b9-hnc8g\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") " pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.334701 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.334604    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:02.482294 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.482250    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"]
Jul 10 16:29:02.487396 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:29:02.487363    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76b737af_0729_447d_babe_5328963939e2.slice/crio-f949a87d81effa1fcb0747dcf85659a4b13f2a1cb67a0c118cf2d39be781a32f WatchSource:0}: Error finding container f949a87d81effa1fcb0747dcf85659a4b13f2a1cb67a0c118cf2d39be781a32f: Status 404 returned error can't find the container with id f949a87d81effa1fcb0747dcf85659a4b13f2a1cb67a0c118cf2d39be781a32f
Jul 10 16:29:02.535224 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:02.535187    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g" event={"ID":"76b737af-0729-447d-babe-5328963939e2","Type":"ContainerStarted","Data":"f949a87d81effa1fcb0747dcf85659a4b13f2a1cb67a0c118cf2d39be781a32f"}
Jul 10 16:29:03.541742 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:03.541707    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g" event={"ID":"76b737af-0729-447d-babe-5328963939e2","Type":"ContainerStarted","Data":"c81a37200a7b976ddca556ee12f226175e860b7bc7337b03198a9a4285c7a21f"}
Jul 10 16:29:03.559443 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:03.559272    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g" podStartSLOduration=1.559253367 podStartE2EDuration="1.559253367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:29:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:29:03.558911881 +0000 UTC m=+2030.702445717" watchObservedRunningTime="2026-07-10 16:29:03.559253367 +0000 UTC m=+2030.702787250"
Jul 10 16:29:12.334892 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:12.334852    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:12.335367 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:12.334926    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:12.347803 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:12.347773    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:12.596569 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:12.596480    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:20.522157 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:20.522124    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"]
Jul 10 16:29:21.509917 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:21.509880    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"]
Jul 10 16:29:35.543672 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.543565    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss" podUID="5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" containerName="main" containerID="cri-o://5b85916572aac015704331a367846b23705874cf2410c1d35d319597877182ec" gracePeriod=45
Jul 10 16:29:35.697785 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.697748    2567 generic.go:356] "Generic (PLEG): container finished" podID="5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" containerID="5b85916572aac015704331a367846b23705874cf2410c1d35d319597877182ec" exitCode=0
Jul 10 16:29:35.697976 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.697818    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss" event={"ID":"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85","Type":"ContainerDied","Data":"5b85916572aac015704331a367846b23705874cf2410c1d35d319597877182ec"}
Jul 10 16:29:35.811019 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.810994    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:29:35.860895 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.860855    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tmp-dir\") pod \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") "
Jul 10 16:29:35.861104 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.860907    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-dshm\") pod \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") "
Jul 10 16:29:35.861104 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.860933    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbgqb\" (UniqueName: \"kubernetes.io/projected/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-kube-api-access-dbgqb\") pod \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") "
Jul 10 16:29:35.861104 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.860968    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-home\") pod \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") "
Jul 10 16:29:35.861104 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.861014    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-model-cache\") pod \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") "
Jul 10 16:29:35.861104 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.861044    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tls-certs\") pod \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\" (UID: \"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85\") "
Jul 10 16:29:35.861359 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.861241    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" (UID: "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:29:35.861359 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.861326    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-model-cache" (OuterVolumeSpecName: "model-cache") pod "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" (UID: "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:29:35.861359 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.861343    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-home" (OuterVolumeSpecName: "home") pod "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" (UID: "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:29:35.861515 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.861442    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tmp-dir\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:35.863522 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.863491    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" (UID: "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 16:29:35.863640 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.863496    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-kube-api-access-dbgqb" (OuterVolumeSpecName: "kube-api-access-dbgqb") pod "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" (UID: "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85"). InnerVolumeSpecName "kube-api-access-dbgqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:29:35.863640 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.863547    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-dshm" (OuterVolumeSpecName: "dshm") pod "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" (UID: "5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:29:35.962385 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.962323    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-dshm\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:35.962385 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.962377    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbgqb\" (UniqueName: \"kubernetes.io/projected/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-kube-api-access-dbgqb\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:35.962385 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.962390    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-home\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:35.962385 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.962398    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-model-cache\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:35.962385 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:35.962422    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85-tls-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:36.531043 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.530970    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g" podUID="76b737af-0729-447d-babe-5328963939e2" containerName="main" containerID="cri-o://c81a37200a7b976ddca556ee12f226175e860b7bc7337b03198a9a4285c7a21f" gracePeriod=45
Jul 10 16:29:36.707386 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.707354    2567 generic.go:356] "Generic (PLEG): container finished" podID="76b737af-0729-447d-babe-5328963939e2" containerID="c81a37200a7b976ddca556ee12f226175e860b7bc7337b03198a9a4285c7a21f" exitCode=0
Jul 10 16:29:36.707826 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.707439    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g" event={"ID":"76b737af-0729-447d-babe-5328963939e2","Type":"ContainerDied","Data":"c81a37200a7b976ddca556ee12f226175e860b7bc7337b03198a9a4285c7a21f"}
Jul 10 16:29:36.708849 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.708826    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss" event={"ID":"5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85","Type":"ContainerDied","Data":"33cc87072afd344b12cf8a16438f27b01591cf1927e6248e9b01406b012b78eb"}
Jul 10 16:29:36.708958 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.708874    2567 scope.go:117] "RemoveContainer" containerID="5b85916572aac015704331a367846b23705874cf2410c1d35d319597877182ec"
Jul 10 16:29:36.708958 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.708909    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"
Jul 10 16:29:36.734326 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.734286    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"]
Jul 10 16:29:36.738108 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.738078    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-0a40f1/test-model-0a40f1-kserve-6cbbb957c7-rfkss"]
Jul 10 16:29:36.792962 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.792893    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:36.871705 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.871666    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xl62n\" (UniqueName: \"kubernetes.io/projected/76b737af-0729-447d-babe-5328963939e2-kube-api-access-xl62n\") pod \"76b737af-0729-447d-babe-5328963939e2\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") "
Jul 10 16:29:36.871899 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.871720    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-tmp-dir\") pod \"76b737af-0729-447d-babe-5328963939e2\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") "
Jul 10 16:29:36.871899 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.871749    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-home\") pod \"76b737af-0729-447d-babe-5328963939e2\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") "
Jul 10 16:29:36.871899 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.871766    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-model-cache\") pod \"76b737af-0729-447d-babe-5328963939e2\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") "
Jul 10 16:29:36.871899 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.871835    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-dshm\") pod \"76b737af-0729-447d-babe-5328963939e2\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") "
Jul 10 16:29:36.871899 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.871868    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76b737af-0729-447d-babe-5328963939e2-tls-certs\") pod \"76b737af-0729-447d-babe-5328963939e2\" (UID: \"76b737af-0729-447d-babe-5328963939e2\") "
Jul 10 16:29:36.872157 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.872066    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "76b737af-0729-447d-babe-5328963939e2" (UID: "76b737af-0729-447d-babe-5328963939e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:29:36.872157 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.872095    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-home" (OuterVolumeSpecName: "home") pod "76b737af-0729-447d-babe-5328963939e2" (UID: "76b737af-0729-447d-babe-5328963939e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:29:36.872232 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.872208    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "76b737af-0729-447d-babe-5328963939e2" (UID: "76b737af-0729-447d-babe-5328963939e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:29:36.874305 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.874265    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b737af-0729-447d-babe-5328963939e2-kube-api-access-xl62n" (OuterVolumeSpecName: "kube-api-access-xl62n") pod "76b737af-0729-447d-babe-5328963939e2" (UID: "76b737af-0729-447d-babe-5328963939e2"). InnerVolumeSpecName "kube-api-access-xl62n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:29:36.874448 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.874364    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-dshm" (OuterVolumeSpecName: "dshm") pod "76b737af-0729-447d-babe-5328963939e2" (UID: "76b737af-0729-447d-babe-5328963939e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:29:36.874448 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.874434    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76b737af-0729-447d-babe-5328963939e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "76b737af-0729-447d-babe-5328963939e2" (UID: "76b737af-0729-447d-babe-5328963939e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 16:29:36.972898 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.972851    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xl62n\" (UniqueName: \"kubernetes.io/projected/76b737af-0729-447d-babe-5328963939e2-kube-api-access-xl62n\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:36.972898 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.972895    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-tmp-dir\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:36.972898 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.972907    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-home\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:36.972898 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.972916    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-model-cache\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:36.973166 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.972923    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/76b737af-0729-447d-babe-5328963939e2-dshm\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:36.973166 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:36.972931    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/76b737af-0729-447d-babe-5328963939e2-tls-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:29:37.404833 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:37.404802    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" path="/var/lib/kubelet/pods/5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85/volumes"
Jul 10 16:29:37.714604 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:37.714575    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"
Jul 10 16:29:37.715079 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:37.714603    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g" event={"ID":"76b737af-0729-447d-babe-5328963939e2","Type":"ContainerDied","Data":"f949a87d81effa1fcb0747dcf85659a4b13f2a1cb67a0c118cf2d39be781a32f"}
Jul 10 16:29:37.715079 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:37.714655    2567 scope.go:117] "RemoveContainer" containerID="c81a37200a7b976ddca556ee12f226175e860b7bc7337b03198a9a4285c7a21f"
Jul 10 16:29:37.730836 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:37.730807    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"]
Jul 10 16:29:37.735009 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:37.734980    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-efe9d3/test-model-efe9d3-kserve-d896f86b9-hnc8g"]
Jul 10 16:29:39.405804 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:29:39.405775    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76b737af-0729-447d-babe-5328963939e2" path="/var/lib/kubelet/pods/76b737af-0729-447d-babe-5328963939e2/volumes"
Jul 10 16:30:11.580954 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:11.580917    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"]
Jul 10 16:30:11.581586 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:11.581246    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy" containerID="cri-o://323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c" gracePeriod=30
Jul 10 16:30:16.851863 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.851786    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:30:16.905586 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.905536    2567 generic.go:356] "Generic (PLEG): container finished" podID="7ceda867-185d-4c19-ab62-4eca78a60014" containerID="323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c" exitCode=0
Jul 10 16:30:16.905771 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.905590    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"
Jul 10 16:30:16.905771 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.905605    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" event={"ID":"7ceda867-185d-4c19-ab62-4eca78a60014","Type":"ContainerDied","Data":"323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c"}
Jul 10 16:30:16.905771 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.905675    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24" event={"ID":"7ceda867-185d-4c19-ab62-4eca78a60014","Type":"ContainerDied","Data":"130002887f0c8968640b92b43bacd04ac27d16693eeabadf732ea1a5a5ffbf6e"}
Jul 10 16:30:16.905771 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.905693    2567 scope.go:117] "RemoveContainer" containerID="323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c"
Jul 10 16:30:16.914985 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.914966    2567 scope.go:117] "RemoveContainer" containerID="036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f"
Jul 10 16:30:16.923851 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.923827    2567 scope.go:117] "RemoveContainer" containerID="323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c"
Jul 10 16:30:16.924204 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:30:16.924179    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c\": container with ID starting with 323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c not found: ID does not exist" containerID="323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c"
Jul 10 16:30:16.924250 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.924217    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c"} err="failed to get container status \"323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c\": rpc error: code = NotFound desc = could not find container \"323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c\": container with ID starting with 323796e01ab4a498ea995e465659e5a7543e5d114b268d3f1b384edbe183ae0c not found: ID does not exist"
Jul 10 16:30:16.924250 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.924237    2567 scope.go:117] "RemoveContainer" containerID="036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f"
Jul 10 16:30:16.924588 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:30:16.924565    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f\": container with ID starting with 036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f not found: ID does not exist" containerID="036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f"
Jul 10 16:30:16.924859 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.924593    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f"} err="failed to get container status \"036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f\": rpc error: code = NotFound desc = could not find container \"036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f\": container with ID starting with 036442a41bd76846911d8829f00ab2a56470da31dcf0cb5910639108b717248f not found: ID does not exist"
Jul 10 16:30:16.948995 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.948963    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-socket\") pod \"7ceda867-185d-4c19-ab62-4eca78a60014\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") "
Jul 10 16:30:16.948995 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949001    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-istio-token\") pod \"7ceda867-185d-4c19-ab62-4eca78a60014\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") "
Jul 10 16:30:16.949206 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949047    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ceda867-185d-4c19-ab62-4eca78a60014-istiod-ca-cert\") pod \"7ceda867-185d-4c19-ab62-4eca78a60014\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") "
Jul 10 16:30:16.949206 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949066    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-certs\") pod \"7ceda867-185d-4c19-ab62-4eca78a60014\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") "
Jul 10 16:30:16.949206 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949094    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ceda867-185d-4c19-ab62-4eca78a60014-istio-podinfo\") pod \"7ceda867-185d-4c19-ab62-4eca78a60014\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") "
Jul 10 16:30:16.949206 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949129    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-credential-socket\") pod \"7ceda867-185d-4c19-ab62-4eca78a60014\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") "
Jul 10 16:30:16.949206 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949169    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjxtf\" (UniqueName: \"kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-kube-api-access-mjxtf\") pod \"7ceda867-185d-4c19-ab62-4eca78a60014\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") "
Jul 10 16:30:16.949206 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949195    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-envoy\") pod \"7ceda867-185d-4c19-ab62-4eca78a60014\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") "
Jul 10 16:30:16.949521 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949230    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-data\") pod \"7ceda867-185d-4c19-ab62-4eca78a60014\" (UID: \"7ceda867-185d-4c19-ab62-4eca78a60014\") "
Jul 10 16:30:16.949521 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949325    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ceda867-185d-4c19-ab62-4eca78a60014" (UID: "7ceda867-185d-4c19-ab62-4eca78a60014"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:30:16.949521 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949395    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ceda867-185d-4c19-ab62-4eca78a60014" (UID: "7ceda867-185d-4c19-ab62-4eca78a60014"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:30:16.949666 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949584    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ceda867-185d-4c19-ab62-4eca78a60014" (UID: "7ceda867-185d-4c19-ab62-4eca78a60014"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:30:16.949666 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949600    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ceda867-185d-4c19-ab62-4eca78a60014-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ceda867-185d-4c19-ab62-4eca78a60014" (UID: "7ceda867-185d-4c19-ab62-4eca78a60014"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 16:30:16.949666 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949643    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:16.949666 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.949665    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-workload-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:16.950513 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.950485    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ceda867-185d-4c19-ab62-4eca78a60014" (UID: "7ceda867-185d-4c19-ab62-4eca78a60014"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:30:16.951671 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.951647    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ceda867-185d-4c19-ab62-4eca78a60014" (UID: "7ceda867-185d-4c19-ab62-4eca78a60014"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:30:16.951671 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.951653    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-kube-api-access-mjxtf" (OuterVolumeSpecName: "kube-api-access-mjxtf") pod "7ceda867-185d-4c19-ab62-4eca78a60014" (UID: "7ceda867-185d-4c19-ab62-4eca78a60014"). InnerVolumeSpecName "kube-api-access-mjxtf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:30:16.951851 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.951833    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ceda867-185d-4c19-ab62-4eca78a60014" (UID: "7ceda867-185d-4c19-ab62-4eca78a60014"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:30:16.951963 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:16.951942    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ceda867-185d-4c19-ab62-4eca78a60014-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ceda867-185d-4c19-ab62-4eca78a60014" (UID: "7ceda867-185d-4c19-ab62-4eca78a60014"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 16:30:17.050662 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:17.050628    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-istio-token\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:17.050662 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:17.050659    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ceda867-185d-4c19-ab62-4eca78a60014-istiod-ca-cert\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:17.050662 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:17.050669    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ceda867-185d-4c19-ab62-4eca78a60014-istio-podinfo\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:17.050910 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:17.050678    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-credential-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:17.050910 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:17.050689    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjxtf\" (UniqueName: \"kubernetes.io/projected/7ceda867-185d-4c19-ab62-4eca78a60014-kube-api-access-mjxtf\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:17.050910 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:17.050698    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-envoy\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:17.050910 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:17.050706    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ceda867-185d-4c19-ab62-4eca78a60014-istio-data\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:17.229544 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:17.229511    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"]
Jul 10 16:30:17.237208 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:17.237178    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-efe9d3-openshift-default-6d4d68567f-vsv24"]
Jul 10 16:30:17.404962 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:17.404931    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" path="/var/lib/kubelet/pods/7ceda867-185d-4c19-ab62-4eca78a60014/volumes"
Jul 10 16:30:21.127999 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.127958    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"]
Jul 10 16:30:21.128504 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.128341    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy"
Jul 10 16:30:21.128504 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.128353    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy"
Jul 10 16:30:21.128504 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.128363    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" containerName="main"
Jul 10 16:30:21.128504 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.128368    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" containerName="main"
Jul 10 16:30:21.128504 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.128377    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy"
Jul 10 16:30:21.128504 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.128383    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy"
Jul 10 16:30:21.128504 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.128399    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76b737af-0729-447d-babe-5328963939e2" containerName="main"
Jul 10 16:30:21.128504 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.128418    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="76b737af-0729-447d-babe-5328963939e2" containerName="main"
Jul 10 16:30:21.128504 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.128474    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="76b737af-0729-447d-babe-5328963939e2" containerName="main"
Jul 10 16:30:21.128504 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.128483    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy"
Jul 10 16:30:21.128504 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.128493    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ce5b52d-9cfb-434c-9ad6-5ee91b3b2c85" containerName="main"
Jul 10 16:30:21.133138 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.133113    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.135581 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.135555    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1375a2-openshift-default-dockercfg-s865n\""
Jul 10 16:30:21.144015 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.143987    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"]
Jul 10 16:30:21.187539 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.187499    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlzl2\" (UniqueName: \"kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-kube-api-access-wlzl2\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.187539 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.187548    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77b612c6-646a-4de9-9328-d99d5a36112a-istio-podinfo\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.187830 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.187611    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-envoy\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.187830 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.187700    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77b612c6-646a-4de9-9328-d99d5a36112a-istiod-ca-cert\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.187830 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.187734    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-certs\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.187830 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.187750    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-istio-token\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.187830 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.187775    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-credential-socket\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.187830 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.187798    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-data\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.188031 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.187871    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-socket\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289089 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289049    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-envoy\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289279 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289100    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77b612c6-646a-4de9-9328-d99d5a36112a-istiod-ca-cert\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289279 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-certs\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289279 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289243    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-istio-token\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289279 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289273    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-credential-socket\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289515 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289289    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-data\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289515 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289330    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-socket\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289515 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289423    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlzl2\" (UniqueName: \"kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-kube-api-access-wlzl2\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289515 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289471    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77b612c6-646a-4de9-9328-d99d5a36112a-istio-podinfo\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289717 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289581    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-certs\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289775 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289726    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-data\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289775 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289724    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-credential-socket\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289885 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289845    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77b612c6-646a-4de9-9328-d99d5a36112a-istiod-ca-cert\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.289885 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.289864    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-socket\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.291657 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.291635    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-envoy\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.292072 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.292045    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77b612c6-646a-4de9-9328-d99d5a36112a-istio-podinfo\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.302857 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.302820    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlzl2\" (UniqueName: \"kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-kube-api-access-wlzl2\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.304758 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.304736    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-istio-token\") pod \"e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") " pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.444661 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.444560    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:21.611268 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.611231    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"]
Jul 10 16:30:21.614068 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:30:21.614041    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b612c6_646a_4de9_9328_d99d5a36112a.slice/crio-711ddd0d0c068791f5e6d5288279630c4505502b9ba3ac626550cdfbbd8a08f1 WatchSource:0}: Error finding container 711ddd0d0c068791f5e6d5288279630c4505502b9ba3ac626550cdfbbd8a08f1: Status 404 returned error can't find the container with id 711ddd0d0c068791f5e6d5288279630c4505502b9ba3ac626550cdfbbd8a08f1
Jul 10 16:30:21.616205 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.616171    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:30:21.616316 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.616248    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:30:21.616316 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.616293    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:30:21.931780 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.931739    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" event={"ID":"77b612c6-646a-4de9-9328-d99d5a36112a","Type":"ContainerStarted","Data":"81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d"}
Jul 10 16:30:21.931992 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.931805    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" event={"ID":"77b612c6-646a-4de9-9328-d99d5a36112a","Type":"ContainerStarted","Data":"711ddd0d0c068791f5e6d5288279630c4505502b9ba3ac626550cdfbbd8a08f1"}
Jul 10 16:30:21.953196 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:21.953138    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" podStartSLOduration=0.95312072 podStartE2EDuration="953.12072ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:30:21.950817579 +0000 UTC m=+2109.094351420" watchObservedRunningTime="2026-07-10 16:30:21.95312072 +0000 UTC m=+2109.096654583"
Jul 10 16:30:22.445495 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:22.445455    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:22.450487 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:22.450456    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:22.937116 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:22.937035    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:22.938163 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:22.938144    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:42.033103 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:42.033065    2567 generic.go:356] "Generic (PLEG): container finished" podID="77b612c6-646a-4de9-9328-d99d5a36112a" containerID="81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d" exitCode=0
Jul 10 16:30:42.033703 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:42.033150    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" event={"ID":"77b612c6-646a-4de9-9328-d99d5a36112a","Type":"ContainerDied","Data":"81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d"}
Jul 10 16:30:42.033777 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:42.033721    2567 scope.go:117] "RemoveContainer" containerID="81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d"
Jul 10 16:30:42.034810 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:42.034783    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:30:42.034870 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:42.034854    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:30:42.034908 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:42.034900    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:30:43.040159 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:43.040128    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" event={"ID":"77b612c6-646a-4de9-9328-d99d5a36112a","Type":"ContainerStarted","Data":"a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a"}
Jul 10 16:30:43.445069 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:43.444974    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:43.445340 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:43.445309    2567 patch_prober.go:28] interesting pod/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 10 16:30:43.445428 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:43.445385    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 10 16:30:44.445806 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:44.445768    2567 patch_prober.go:28] interesting pod/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 10 16:30:44.446183 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:44.445827    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 10 16:30:45.445576 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:45.445539    2567 patch_prober.go:28] interesting pod/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 10 16:30:45.445774 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:45.445598    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 10 16:30:46.445754 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:46.445705    2567 patch_prober.go:28] interesting pod/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 10 16:30:46.446236 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:46.445780    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 10 16:30:47.448342 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:47.448310    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:47.448792 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:47.448635    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:47.449519 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:47.449501    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:50.706623 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:50.706583    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"]
Jul 10 16:30:50.707020 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:50.706774    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" containerName="istio-proxy" containerID="cri-o://a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a" gracePeriod=30
Jul 10 16:30:55.991650 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:55.991619    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:56.105860 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.105827    2567 generic.go:356] "Generic (PLEG): container finished" podID="77b612c6-646a-4de9-9328-d99d5a36112a" containerID="a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a" exitCode=0
Jul 10 16:30:56.106011 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.105867    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" event={"ID":"77b612c6-646a-4de9-9328-d99d5a36112a","Type":"ContainerDied","Data":"a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a"}
Jul 10 16:30:56.106011 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.105875    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"
Jul 10 16:30:56.106011 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.105912    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z" event={"ID":"77b612c6-646a-4de9-9328-d99d5a36112a","Type":"ContainerDied","Data":"711ddd0d0c068791f5e6d5288279630c4505502b9ba3ac626550cdfbbd8a08f1"}
Jul 10 16:30:56.106011 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.105941    2567 scope.go:117] "RemoveContainer" containerID="a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a"
Jul 10 16:30:56.112121 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112095    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77b612c6-646a-4de9-9328-d99d5a36112a-istio-podinfo\") pod \"77b612c6-646a-4de9-9328-d99d5a36112a\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") "
Jul 10 16:30:56.112249 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112138    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-certs\") pod \"77b612c6-646a-4de9-9328-d99d5a36112a\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") "
Jul 10 16:30:56.112249 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112156    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-data\") pod \"77b612c6-646a-4de9-9328-d99d5a36112a\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") "
Jul 10 16:30:56.112249 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112197    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-envoy\") pod \"77b612c6-646a-4de9-9328-d99d5a36112a\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") "
Jul 10 16:30:56.112426 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112270    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-credential-socket\") pod \"77b612c6-646a-4de9-9328-d99d5a36112a\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") "
Jul 10 16:30:56.112426 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112306    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77b612c6-646a-4de9-9328-d99d5a36112a-istiod-ca-cert\") pod \"77b612c6-646a-4de9-9328-d99d5a36112a\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") "
Jul 10 16:30:56.112426 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112330    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-istio-token\") pod \"77b612c6-646a-4de9-9328-d99d5a36112a\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") "
Jul 10 16:30:56.112426 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112363    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-socket\") pod \"77b612c6-646a-4de9-9328-d99d5a36112a\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") "
Jul 10 16:30:56.112426 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112391    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlzl2\" (UniqueName: \"kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-kube-api-access-wlzl2\") pod \"77b612c6-646a-4de9-9328-d99d5a36112a\" (UID: \"77b612c6-646a-4de9-9328-d99d5a36112a\") "
Jul 10 16:30:56.112679 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112544    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "77b612c6-646a-4de9-9328-d99d5a36112a" (UID: "77b612c6-646a-4de9-9328-d99d5a36112a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:30:56.112899 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112770    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:56.112899 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112824    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "77b612c6-646a-4de9-9328-d99d5a36112a" (UID: "77b612c6-646a-4de9-9328-d99d5a36112a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:30:56.113043 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112966    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "77b612c6-646a-4de9-9328-d99d5a36112a" (UID: "77b612c6-646a-4de9-9328-d99d5a36112a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:30:56.113043 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.112995    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77b612c6-646a-4de9-9328-d99d5a36112a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "77b612c6-646a-4de9-9328-d99d5a36112a" (UID: "77b612c6-646a-4de9-9328-d99d5a36112a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 16:30:56.113731 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.113706    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-data" (OuterVolumeSpecName: "istio-data") pod "77b612c6-646a-4de9-9328-d99d5a36112a" (UID: "77b612c6-646a-4de9-9328-d99d5a36112a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:30:56.115177 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.115141    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77b612c6-646a-4de9-9328-d99d5a36112a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "77b612c6-646a-4de9-9328-d99d5a36112a" (UID: "77b612c6-646a-4de9-9328-d99d5a36112a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 16:30:56.115177 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.115152    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-istio-token" (OuterVolumeSpecName: "istio-token") pod "77b612c6-646a-4de9-9328-d99d5a36112a" (UID: "77b612c6-646a-4de9-9328-d99d5a36112a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:30:56.115328 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.115308    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-kube-api-access-wlzl2" (OuterVolumeSpecName: "kube-api-access-wlzl2") pod "77b612c6-646a-4de9-9328-d99d5a36112a" (UID: "77b612c6-646a-4de9-9328-d99d5a36112a"). InnerVolumeSpecName "kube-api-access-wlzl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:30:56.115376 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.115347    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "77b612c6-646a-4de9-9328-d99d5a36112a" (UID: "77b612c6-646a-4de9-9328-d99d5a36112a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:30:56.116266 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.116249    2567 scope.go:117] "RemoveContainer" containerID="81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d"
Jul 10 16:30:56.134583 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.134556    2567 scope.go:117] "RemoveContainer" containerID="a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a"
Jul 10 16:30:56.134938 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:30:56.134913    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a\": container with ID starting with a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a not found: ID does not exist" containerID="a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a"
Jul 10 16:30:56.135011 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.134945    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a"} err="failed to get container status \"a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a\": rpc error: code = NotFound desc = could not find container \"a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a\": container with ID starting with a42023457a830898aff351b01c66ad477cb9b4f6cbb399b1a679361312e4437a not found: ID does not exist"
Jul 10 16:30:56.135011 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.134970    2567 scope.go:117] "RemoveContainer" containerID="81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d"
Jul 10 16:30:56.135245 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:30:56.135226    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d\": container with ID starting with 81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d not found: ID does not exist" containerID="81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d"
Jul 10 16:30:56.135300 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.135256    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d"} err="failed to get container status \"81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d\": rpc error: code = NotFound desc = could not find container \"81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d\": container with ID starting with 81262cdb1117745fd0a9149ddfeb636253903107314e5dd2ec2297848bee9b1d not found: ID does not exist"
Jul 10 16:30:56.213985 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.213924    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77b612c6-646a-4de9-9328-d99d5a36112a-istio-podinfo\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:56.213985 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.213979    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-data\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:56.213985 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.213989    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-istio-envoy\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:56.213985 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.213997    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-credential-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:56.214255 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.214006    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77b612c6-646a-4de9-9328-d99d5a36112a-istiod-ca-cert\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:56.214255 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.214014    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-istio-token\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:56.214255 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.214021    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b612c6-646a-4de9-9328-d99d5a36112a-workload-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:56.214255 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.214030    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlzl2\" (UniqueName: \"kubernetes.io/projected/77b612c6-646a-4de9-9328-d99d5a36112a-kube-api-access-wlzl2\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:30:56.431183 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.431148    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"]
Jul 10 16:30:56.436382 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:56.436357    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1375a2-openshift-default-f894b4dc9-bcf6z"]
Jul 10 16:30:57.405139 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:57.405103    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" path="/var/lib/kubelet/pods/77b612c6-646a-4de9-9328-d99d5a36112a/volumes"
Jul 10 16:30:59.266269 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.266230    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"]
Jul 10 16:30:59.266695 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.266641    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" containerName="istio-proxy"
Jul 10 16:30:59.266695 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.266655    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" containerName="istio-proxy"
Jul 10 16:30:59.266695 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.266667    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" containerName="istio-proxy"
Jul 10 16:30:59.266695 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.266672    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" containerName="istio-proxy"
Jul 10 16:30:59.266848 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.266725    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" containerName="istio-proxy"
Jul 10 16:30:59.266848 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.266735    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b612c6-646a-4de9-9328-d99d5a36112a" containerName="istio-proxy"
Jul 10 16:30:59.266848 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.266744    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ceda867-185d-4c19-ab62-4eca78a60014" containerName="istio-proxy"
Jul 10 16:30:59.271209 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.271184    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.274196 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.274166    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-466b1a-openshift-default-dockercfg-44ldm\""
Jul 10 16:30:59.283641 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.283608    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"]
Jul 10 16:30:59.344800 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.344765    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-podinfo\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.344800 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.344804    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfkh2\" (UniqueName: \"kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-kube-api-access-xfkh2\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.345064 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.344861    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-socket\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.345064 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.344887    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-token\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.345064 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.344920    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-credential-socket\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.345064 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.344980    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-data\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.345064 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.345018    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/449c4b13-9e17-4ca4-b83f-721aaed4c704-istiod-ca-cert\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.345282 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.345071    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-certs\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.345282 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.345135    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-envoy\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.446179 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.446141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-certs\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.446380 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.446227    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-envoy\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.446380 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.446275    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-podinfo\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.446380 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.446300    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfkh2\" (UniqueName: \"kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-kube-api-access-xfkh2\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.446380 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.446340    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-socket\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.446380 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.446364    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-token\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.446773 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.446447    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-credential-socket\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.446773 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.446527    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-data\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.446773 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.446573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/449c4b13-9e17-4ca4-b83f-721aaed4c704-istiod-ca-cert\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.446773 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.446650    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-certs\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.447024 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.446989    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-credential-socket\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.447116 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.447035    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-socket\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.447282 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.447205    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-data\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.447282 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.447226    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/449c4b13-9e17-4ca4-b83f-721aaed4c704-istiod-ca-cert\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.448893 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.448868    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-envoy\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.448979 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.448954    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-podinfo\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.457813 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.457790    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-token\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.457942 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.457873    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfkh2\" (UniqueName: \"kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-kube-api-access-xfkh2\") pod \"e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") " pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.583432 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.583315    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:30:59.731522 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.731496    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"]
Jul 10 16:30:59.733228 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:30:59.733200    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod449c4b13_9e17_4ca4_b83f_721aaed4c704.slice/crio-a33eee45fbe969d78b6299ac15f63f319b03fc0e21f7e36af05ffc9ddc8583b7 WatchSource:0}: Error finding container a33eee45fbe969d78b6299ac15f63f319b03fc0e21f7e36af05ffc9ddc8583b7: Status 404 returned error can't find the container with id a33eee45fbe969d78b6299ac15f63f319b03fc0e21f7e36af05ffc9ddc8583b7
Jul 10 16:30:59.735337 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.735306    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:30:59.735441 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.735374    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:30:59.735441 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:30:59.735419    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:31:00.127270 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:00.127230    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" event={"ID":"449c4b13-9e17-4ca4-b83f-721aaed4c704","Type":"ContainerStarted","Data":"a6511cfbfda5fa9b10d0a1b904df08ad1ca45456971d257308da147ce6d468a1"}
Jul 10 16:31:00.127507 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:00.127291    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" event={"ID":"449c4b13-9e17-4ca4-b83f-721aaed4c704","Type":"ContainerStarted","Data":"a33eee45fbe969d78b6299ac15f63f319b03fc0e21f7e36af05ffc9ddc8583b7"}
Jul 10 16:31:00.161297 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:00.161241    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" podStartSLOduration=1.161224686 podStartE2EDuration="1.161224686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:30:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:31:00.160165815 +0000 UTC m=+2147.303699654" watchObservedRunningTime="2026-07-10 16:31:00.161224686 +0000 UTC m=+2147.304758574"
Jul 10 16:31:00.584262 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:00.584221    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:31:00.589654 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:00.589625    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:31:01.131890 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:01.131858    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:31:01.132915 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:01.132893    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:31:17.174655 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:17.174606    2567 patch_prober.go:28] interesting pod/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 16:31:17.175056 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:17.174686    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" podUID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 16:31:18.545367 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.545329    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"]
Jul 10 16:31:18.549289 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.549263    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.553138 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.553092    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c32d0d\"/\"default-dockercfg-jxbmd\""
Jul 10 16:31:18.553138 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.553111    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c32d0d\"/\"kube-root-ca.crt\""
Jul 10 16:31:18.553345 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.553091    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c32d0d\"/\"auth-test-model-c32d0d-kserve-self-signed-certs\""
Jul 10 16:31:18.553345 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.553092    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c32d0d\"/\"openshift-service-ca.crt\""
Jul 10 16:31:18.559801 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.559774    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"]
Jul 10 16:31:18.738074 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.738039    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-model-cache\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.738275 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.738084    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-home\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.738275 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.738170    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-dshm\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.738275 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.738207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzxcl\" (UniqueName: \"kubernetes.io/projected/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-kube-api-access-jzxcl\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.738275 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.738243    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tmp-dir\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.738275 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.738263    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tls-certs\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.839507 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.839384    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tmp-dir\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.839507 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.839460    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tls-certs\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.839507 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.839498    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-model-cache\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.839759 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.839531    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-home\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.839759 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.839583    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-dshm\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.839759 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.839616    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzxcl\" (UniqueName: \"kubernetes.io/projected/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-kube-api-access-jzxcl\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.839918 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.839831    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tmp-dir\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.839918 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.839887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-model-cache\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.840024 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.839984    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-home\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.842060 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.842035    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-dshm\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.842162 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.842147    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tls-certs\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.850199 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.850176    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzxcl\" (UniqueName: \"kubernetes.io/projected/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-kube-api-access-jzxcl\") pod \"auth-test-model-c32d0d-kserve-77c9f8487-h7ww7\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") " pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:18.862016 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:18.861985    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:19.014262 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:19.014232    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"]
Jul 10 16:31:19.016732 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:31:19.016692    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod884bc84c_ee50_41e7_a14f_1c1fafcb0f89.slice/crio-0c0a398e86bd0bd6de3bd41abed58952ac9a97de2e1e793d0f7a823a591b9a31 WatchSource:0}: Error finding container 0c0a398e86bd0bd6de3bd41abed58952ac9a97de2e1e793d0f7a823a591b9a31: Status 404 returned error can't find the container with id 0c0a398e86bd0bd6de3bd41abed58952ac9a97de2e1e793d0f7a823a591b9a31
Jul 10 16:31:19.226213 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:19.226170    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7" event={"ID":"884bc84c-ee50-41e7-a14f-1c1fafcb0f89","Type":"ContainerStarted","Data":"e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b"}
Jul 10 16:31:19.226213 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:19.226219    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7" event={"ID":"884bc84c-ee50-41e7-a14f-1c1fafcb0f89","Type":"ContainerStarted","Data":"0c0a398e86bd0bd6de3bd41abed58952ac9a97de2e1e793d0f7a823a591b9a31"}
Jul 10 16:31:19.227858 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:19.227832    2567 generic.go:356] "Generic (PLEG): container finished" podID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerID="a6511cfbfda5fa9b10d0a1b904df08ad1ca45456971d257308da147ce6d468a1" exitCode=0
Jul 10 16:31:19.227975 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:19.227864    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" event={"ID":"449c4b13-9e17-4ca4-b83f-721aaed4c704","Type":"ContainerDied","Data":"a6511cfbfda5fa9b10d0a1b904df08ad1ca45456971d257308da147ce6d468a1"}
Jul 10 16:31:19.228190 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:19.228176    2567 scope.go:117] "RemoveContainer" containerID="a6511cfbfda5fa9b10d0a1b904df08ad1ca45456971d257308da147ce6d468a1"
Jul 10 16:31:19.229146 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:19.229118    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:31:19.229199 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:19.229190    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:31:19.229236 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:19.229228    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 16:31:19.245558 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:19.245450    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7" podStartSLOduration=1.245430004 podStartE2EDuration="1.245430004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:31:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:31:19.244987449 +0000 UTC m=+2166.388521291" watchObservedRunningTime="2026-07-10 16:31:19.245430004 +0000 UTC m=+2166.388963847"
Jul 10 16:31:20.234479 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:20.234447    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" event={"ID":"449c4b13-9e17-4ca4-b83f-721aaed4c704","Type":"ContainerStarted","Data":"0b5841c7b52ce1bbb005f2cff7bfdc1c039f5968fafa8a09b3839cbfaa2464fe"}
Jul 10 16:31:20.584649 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:20.584074    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:31:20.584649 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:20.584227    2567 patch_prober.go:28] interesting pod/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 10 16:31:20.584649 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:20.584272    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" podUID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 10 16:31:21.585124 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:21.584673    2567 patch_prober.go:28] interesting pod/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 10 16:31:21.585124 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:21.584739    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" podUID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 10 16:31:22.584542 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:22.584492    2567 patch_prober.go:28] interesting pod/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 10 16:31:22.584765 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:22.584581    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" podUID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 10 16:31:23.585096 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:23.584591    2567 patch_prober.go:28] interesting pod/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 10 16:31:23.585096 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:23.584662    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" podUID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 10 16:31:24.587658 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:24.587627    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:31:24.588070 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:24.587996    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:31:24.588662 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:24.588646    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:31:28.863025 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:28.862987    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:28.863025 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:28.863030    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:28.875708 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:28.875678    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:29.286755 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:29.286722    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:31:39.638081 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.638044    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"]
Jul 10 16:31:39.641143 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.641122    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.644121 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.644099    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"kube-root-ca.crt\""
Jul 10 16:31:39.645155 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.645134    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"default-dockercfg-k7jrd\""
Jul 10 16:31:39.645210 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.645181    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"openshift-service-ca.crt\""
Jul 10 16:31:39.645287 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.645271    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"auth-test-model-466b1a-kserve-self-signed-certs\""
Jul 10 16:31:39.652328 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.652301    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"]
Jul 10 16:31:39.717881 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.717844    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-home\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.718062 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.717910    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-dshm\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.718062 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.717953    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tls-certs\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.718062 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.717996    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n74d7\" (UniqueName: \"kubernetes.io/projected/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-kube-api-access-n74d7\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.718062 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.718038    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-model-cache\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.718062 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.718063    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tmp-dir\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.818603 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.818570    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-dshm\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.818856 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.818613    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tls-certs\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.818856 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.818646    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n74d7\" (UniqueName: \"kubernetes.io/projected/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-kube-api-access-n74d7\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.818856 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.818690    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-model-cache\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.818856 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.818718    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tmp-dir\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.818856 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.818745    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-home\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.819129 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.819112    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-home\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.819169 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.819140    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-model-cache\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.819221 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.819201    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tmp-dir\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.821097 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.821077    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-dshm\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.821345 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.821328    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tls-certs\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.827736 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.827714    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n74d7\" (UniqueName: \"kubernetes.io/projected/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-kube-api-access-n74d7\") pod \"auth-test-model-466b1a-kserve-56f85bcb95-mhvjv\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") " pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:39.951634 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:39.951598    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:40.099078 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:40.099047    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"]
Jul 10 16:31:40.101165 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:31:40.101134    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31b3918d_e3bc_4166_9c5d_8b2fb907b1cb.slice/crio-e7f52af6d00187eb33e15c5c45cb8621b6ac742460eb3e423c2c398beef98234 WatchSource:0}: Error finding container e7f52af6d00187eb33e15c5c45cb8621b6ac742460eb3e423c2c398beef98234: Status 404 returned error can't find the container with id e7f52af6d00187eb33e15c5c45cb8621b6ac742460eb3e423c2c398beef98234
Jul 10 16:31:40.329954 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:40.329858    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv" event={"ID":"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb","Type":"ContainerStarted","Data":"91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33"}
Jul 10 16:31:40.329954 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:40.329905    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv" event={"ID":"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb","Type":"ContainerStarted","Data":"e7f52af6d00187eb33e15c5c45cb8621b6ac742460eb3e423c2c398beef98234"}
Jul 10 16:31:40.349892 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:40.349827    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv" podStartSLOduration=1.349808307 podStartE2EDuration="1.349808307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:31:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:31:40.348351155 +0000 UTC m=+2187.491884994" watchObservedRunningTime="2026-07-10 16:31:40.349808307 +0000 UTC m=+2187.493342146"
Jul 10 16:31:49.952158 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:49.952121    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:49.952652 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:49.952450    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:49.965116 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:49.965089    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:31:50.395514 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:31:50.395392    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:33:43.572659 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:43.572624    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"]
Jul 10 16:33:44.530884 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:44.530842    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"]
Jul 10 16:33:58.594505 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.594462    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7" podUID="884bc84c-ee50-41e7-a14f-1c1fafcb0f89" containerName="main" containerID="cri-o://e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b" gracePeriod=45
Jul 10 16:33:58.861349 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.861323    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:33:58.976085 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.976049    2567 generic.go:356] "Generic (PLEG): container finished" podID="884bc84c-ee50-41e7-a14f-1c1fafcb0f89" containerID="e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b" exitCode=0
Jul 10 16:33:58.976271 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.976095    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"
Jul 10 16:33:58.976271 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.976144    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7" event={"ID":"884bc84c-ee50-41e7-a14f-1c1fafcb0f89","Type":"ContainerDied","Data":"e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b"}
Jul 10 16:33:58.976271 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.976188    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7" event={"ID":"884bc84c-ee50-41e7-a14f-1c1fafcb0f89","Type":"ContainerDied","Data":"0c0a398e86bd0bd6de3bd41abed58952ac9a97de2e1e793d0f7a823a591b9a31"}
Jul 10 16:33:58.976271 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.976211    2567 scope.go:117] "RemoveContainer" containerID="e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b"
Jul 10 16:33:58.985245 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.985225    2567 scope.go:117] "RemoveContainer" containerID="e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b"
Jul 10 16:33:58.985590 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:33:58.985568    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b\": container with ID starting with e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b not found: ID does not exist" containerID="e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b"
Jul 10 16:33:58.985693 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.985603    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b"} err="failed to get container status \"e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b\": rpc error: code = NotFound desc = could not find container \"e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b\": container with ID starting with e8c8c22c79be92a92757eab531558b5bfbd65eef932ce8e7bf0984f4bcdace3b not found: ID does not exist"
Jul 10 16:33:58.986940 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.986920    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-dshm\") pod \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") "
Jul 10 16:33:58.987009 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.986969    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tls-certs\") pod \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") "
Jul 10 16:33:58.987060 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.987023    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tmp-dir\") pod \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") "
Jul 10 16:33:58.987114 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.987060    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-model-cache\") pod \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") "
Jul 10 16:33:58.987168 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.987115    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-home\") pod \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") "
Jul 10 16:33:58.987168 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.987146    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzxcl\" (UniqueName: \"kubernetes.io/projected/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-kube-api-access-jzxcl\") pod \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\" (UID: \"884bc84c-ee50-41e7-a14f-1c1fafcb0f89\") "
Jul 10 16:33:58.987507 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.987316    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "884bc84c-ee50-41e7-a14f-1c1fafcb0f89" (UID: "884bc84c-ee50-41e7-a14f-1c1fafcb0f89"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:33:58.987507 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.987355    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-home" (OuterVolumeSpecName: "home") pod "884bc84c-ee50-41e7-a14f-1c1fafcb0f89" (UID: "884bc84c-ee50-41e7-a14f-1c1fafcb0f89"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:33:58.987507 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.987371    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-model-cache" (OuterVolumeSpecName: "model-cache") pod "884bc84c-ee50-41e7-a14f-1c1fafcb0f89" (UID: "884bc84c-ee50-41e7-a14f-1c1fafcb0f89"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:33:58.987507 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.987439    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tmp-dir\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:33:58.989554 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.989517    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-dshm" (OuterVolumeSpecName: "dshm") pod "884bc84c-ee50-41e7-a14f-1c1fafcb0f89" (UID: "884bc84c-ee50-41e7-a14f-1c1fafcb0f89"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:33:58.989635 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.989613    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-kube-api-access-jzxcl" (OuterVolumeSpecName: "kube-api-access-jzxcl") pod "884bc84c-ee50-41e7-a14f-1c1fafcb0f89" (UID: "884bc84c-ee50-41e7-a14f-1c1fafcb0f89"). InnerVolumeSpecName "kube-api-access-jzxcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:33:58.989635 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:58.989620    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "884bc84c-ee50-41e7-a14f-1c1fafcb0f89" (UID: "884bc84c-ee50-41e7-a14f-1c1fafcb0f89"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 16:33:59.088070 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.088040    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-tls-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:33:59.088070 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.088066    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-model-cache\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:33:59.088070 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.088074    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-home\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:33:59.088370 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.088085    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzxcl\" (UniqueName: \"kubernetes.io/projected/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-kube-api-access-jzxcl\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:33:59.088370 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.088093    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/884bc84c-ee50-41e7-a14f-1c1fafcb0f89-dshm\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:33:59.297506 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.297471    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"]
Jul 10 16:33:59.302338 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.302313    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/auth-test-model-c32d0d-kserve-77c9f8487-h7ww7"]
Jul 10 16:33:59.411431 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.408738    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="884bc84c-ee50-41e7-a14f-1c1fafcb0f89" path="/var/lib/kubelet/pods/884bc84c-ee50-41e7-a14f-1c1fafcb0f89/volumes"
Jul 10 16:33:59.552334 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.552208    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv" podUID="31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" containerName="main" containerID="cri-o://91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33" gracePeriod=45
Jul 10 16:33:59.815017 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.814948    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:33:59.846258 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.846220    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"]
Jul 10 16:33:59.846786 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.846767    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="884bc84c-ee50-41e7-a14f-1c1fafcb0f89" containerName="main"
Jul 10 16:33:59.846834 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.846791    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="884bc84c-ee50-41e7-a14f-1c1fafcb0f89" containerName="main"
Jul 10 16:33:59.846834 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.846803    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" containerName="main"
Jul 10 16:33:59.846834 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.846811    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" containerName="main"
Jul 10 16:33:59.846956 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.846944    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" containerName="main"
Jul 10 16:33:59.846988 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.846962    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="884bc84c-ee50-41e7-a14f-1c1fafcb0f89" containerName="main"
Jul 10 16:33:59.850317 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.850298    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:33:59.852911 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.852888    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c32d0d\"/\"openshift-service-ca.crt\""
Jul 10 16:33:59.853098 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.852887    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c32d0d\"/\"kube-root-ca.crt\""
Jul 10 16:33:59.853098 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.852983    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c32d0d\"/\"sub-test-model-c32d0d-kserve-self-signed-certs\""
Jul 10 16:33:59.853098 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.853045    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c32d0d\"/\"default-dockercfg-jxbmd\""
Jul 10 16:33:59.856530 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.856503    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"]
Jul 10 16:33:59.983315 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.983278    2567 generic.go:356] "Generic (PLEG): container finished" podID="31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" containerID="91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33" exitCode=0
Jul 10 16:33:59.983540 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.983321    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"
Jul 10 16:33:59.983540 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.983367    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv" event={"ID":"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb","Type":"ContainerDied","Data":"91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33"}
Jul 10 16:33:59.983540 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.983438    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv" event={"ID":"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb","Type":"ContainerDied","Data":"e7f52af6d00187eb33e15c5c45cb8621b6ac742460eb3e423c2c398beef98234"}
Jul 10 16:33:59.983540 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.983464    2567 scope.go:117] "RemoveContainer" containerID="91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33"
Jul 10 16:33:59.992188 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.992169    2567 scope.go:117] "RemoveContainer" containerID="91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33"
Jul 10 16:33:59.992488 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:33:59.992462    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33\": container with ID starting with 91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33 not found: ID does not exist" containerID="91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33"
Jul 10 16:33:59.992589 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.992498    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33"} err="failed to get container status \"91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33\": rpc error: code = NotFound desc = could not find container \"91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33\": container with ID starting with 91e691a90f54409e954c0326cf68223f44c2b86e82acd0865b5abb512928ce33 not found: ID does not exist"
Jul 10 16:33:59.994797 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.994776    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-home\") pod \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") "
Jul 10 16:33:59.994911 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.994806    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n74d7\" (UniqueName: \"kubernetes.io/projected/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-kube-api-access-n74d7\") pod \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") "
Jul 10 16:33:59.994911 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.994873    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tls-certs\") pod \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") "
Jul 10 16:33:59.995023 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.994965    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-dshm\") pod \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") "
Jul 10 16:33:59.995023 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.994989    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-model-cache\") pod \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") "
Jul 10 16:33:59.995023 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995011    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tmp-dir\") pod \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\" (UID: \"31b3918d-e3bc-4166-9c5d-8b2fb907b1cb\") "
Jul 10 16:33:59.995169 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995065    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-home" (OuterVolumeSpecName: "home") pod "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" (UID: "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:33:59.995169 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995128    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-model-cache\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:33:59.995273 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995203    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-tmp-dir\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:33:59.995273 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995229    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fbmc\" (UniqueName: \"kubernetes.io/projected/362a595f-3f9c-486a-9267-c98f7c76db27-kube-api-access-7fbmc\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:33:59.995378 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995295    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-model-cache" (OuterVolumeSpecName: "model-cache") pod "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" (UID: "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:33:59.995378 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995283    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-home\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:33:59.995512 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995457    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-dshm\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:33:59.995569 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995523    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362a595f-3f9c-486a-9267-c98f7c76db27-tls-certs\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:33:59.995623 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995614    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-model-cache\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:33:59.995733 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995630    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-home\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:33:59.995901 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.995849    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" (UID: "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:33:59.997344 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.997315    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-kube-api-access-n74d7" (OuterVolumeSpecName: "kube-api-access-n74d7") pod "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" (UID: "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb"). InnerVolumeSpecName "kube-api-access-n74d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:33:59.997481 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.997353    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" (UID: "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 16:33:59.997481 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:33:59.997395    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-dshm" (OuterVolumeSpecName: "dshm") pod "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" (UID: "31b3918d-e3bc-4166-9c5d-8b2fb907b1cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:34:00.096193 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362a595f-3f9c-486a-9267-c98f7c76db27-tls-certs\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.096193 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096146    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-model-cache\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.096193 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096186    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-tmp-dir\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.096528 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096203    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fbmc\" (UniqueName: \"kubernetes.io/projected/362a595f-3f9c-486a-9267-c98f7c76db27-kube-api-access-7fbmc\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.096528 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-home\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.096528 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096274    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-dshm\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.096528 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096325    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n74d7\" (UniqueName: \"kubernetes.io/projected/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-kube-api-access-n74d7\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:34:00.096528 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096338    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tls-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:34:00.096528 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096347    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-dshm\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:34:00.096528 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096354    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb-tmp-dir\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:34:00.096803 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096710    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-model-cache\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.096803 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096733    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-tmp-dir\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.096803 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.096795    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-home\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.098705 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.098685    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-dshm\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.099010 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.098978    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362a595f-3f9c-486a-9267-c98f7c76db27-tls-certs\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.104449 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.104398    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fbmc\" (UniqueName: \"kubernetes.io/projected/362a595f-3f9c-486a-9267-c98f7c76db27-kube-api-access-7fbmc\") pod \"sub-test-model-c32d0d-kserve-6d86579c87-vdk5v\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") " pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.160660 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.160616    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:00.306520 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.306477    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"]
Jul 10 16:34:00.309251 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.309225    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"]
Jul 10 16:34:00.311238 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.311213    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/auth-test-model-466b1a-kserve-56f85bcb95-mhvjv"]
Jul 10 16:34:00.311506 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:34:00.311485    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod362a595f_3f9c_486a_9267_c98f7c76db27.slice/crio-8b3d08cb9eebb27bee5526901d77789e56c4b17caedfa09861e0dbe2d4786244 WatchSource:0}: Error finding container 8b3d08cb9eebb27bee5526901d77789e56c4b17caedfa09861e0dbe2d4786244: Status 404 returned error can't find the container with id 8b3d08cb9eebb27bee5526901d77789e56c4b17caedfa09861e0dbe2d4786244
Jul 10 16:34:00.990494 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.990454    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v" event={"ID":"362a595f-3f9c-486a-9267-c98f7c76db27","Type":"ContainerStarted","Data":"843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e"}
Jul 10 16:34:00.990882 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:00.990501    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v" event={"ID":"362a595f-3f9c-486a-9267-c98f7c76db27","Type":"ContainerStarted","Data":"8b3d08cb9eebb27bee5526901d77789e56c4b17caedfa09861e0dbe2d4786244"}
Jul 10 16:34:01.014773 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:01.014724    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v" podStartSLOduration=2.014706584 podStartE2EDuration="2.014706584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:34:01.013991568 +0000 UTC m=+2328.157525411" watchObservedRunningTime="2026-07-10 16:34:01.014706584 +0000 UTC m=+2328.158240457"
Jul 10 16:34:01.405391 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:01.405309    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31b3918d-e3bc-4166-9c5d-8b2fb907b1cb" path="/var/lib/kubelet/pods/31b3918d-e3bc-4166-9c5d-8b2fb907b1cb/volumes"
Jul 10 16:34:10.161286 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:10.161195    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:10.161286 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:10.161250    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:10.174465 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:10.174440    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:11.048699 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:11.048662    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:34:32.447781 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.447739    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"]
Jul 10 16:34:32.451079 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.451061    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.454652 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.454624    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"sub-test-model-466b1a-kserve-self-signed-certs\""
Jul 10 16:34:32.454803 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.454625    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"openshift-service-ca.crt\""
Jul 10 16:34:32.454803 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.454625    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"default-dockercfg-k7jrd\""
Jul 10 16:34:32.454803 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.454658    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"kube-root-ca.crt\""
Jul 10 16:34:32.459481 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.459450    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"]
Jul 10 16:34:32.585148 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.585110    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e53ae09-78f1-430f-97e3-f06261334817-tls-certs\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.585314 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.585183    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-model-cache\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.585314 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.585208    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-home\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.585314 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.585228    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb9rk\" (UniqueName: \"kubernetes.io/projected/0e53ae09-78f1-430f-97e3-f06261334817-kube-api-access-jb9rk\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.585314 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.585297    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-tmp-dir\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.585314 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.585315    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-dshm\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.686024 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.685984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-home\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.686024 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.686028    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb9rk\" (UniqueName: \"kubernetes.io/projected/0e53ae09-78f1-430f-97e3-f06261334817-kube-api-access-jb9rk\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.686318 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.686078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-tmp-dir\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.686318 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.686103    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-dshm\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.686318 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.686165    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e53ae09-78f1-430f-97e3-f06261334817-tls-certs\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.686318 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.686239    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-model-cache\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.686585 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.686511    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-home\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.686585 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.686547    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-tmp-dir\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.686671 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.686615    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-model-cache\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.688752 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.688720    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-dshm\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.688933 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.688914    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e53ae09-78f1-430f-97e3-f06261334817-tls-certs\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.695836 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.695806    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb9rk\" (UniqueName: \"kubernetes.io/projected/0e53ae09-78f1-430f-97e3-f06261334817-kube-api-access-jb9rk\") pod \"sub-test-model-466b1a-kserve-75554b95f-tll7b\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") " pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.761074 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.760965    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:32.912168 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:32.912140    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"]
Jul 10 16:34:32.914193 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:34:32.914152    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e53ae09_78f1_430f_97e3_f06261334817.slice/crio-2c2cce20ff8e4d9032042a1b0f44ab54f1f9a82d37b0c196284f1fc35dd29a2a WatchSource:0}: Error finding container 2c2cce20ff8e4d9032042a1b0f44ab54f1f9a82d37b0c196284f1fc35dd29a2a: Status 404 returned error can't find the container with id 2c2cce20ff8e4d9032042a1b0f44ab54f1f9a82d37b0c196284f1fc35dd29a2a
Jul 10 16:34:33.140992 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:33.140891    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b" event={"ID":"0e53ae09-78f1-430f-97e3-f06261334817","Type":"ContainerStarted","Data":"1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e"}
Jul 10 16:34:33.140992 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:33.140940    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b" event={"ID":"0e53ae09-78f1-430f-97e3-f06261334817","Type":"ContainerStarted","Data":"2c2cce20ff8e4d9032042a1b0f44ab54f1f9a82d37b0c196284f1fc35dd29a2a"}
Jul 10 16:34:33.162913 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:33.162746    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b" podStartSLOduration=1.162723678 podStartE2EDuration="1.162723678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:34:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:34:33.161908943 +0000 UTC m=+2360.305442782" watchObservedRunningTime="2026-07-10 16:34:33.162723678 +0000 UTC m=+2360.306257508"
Jul 10 16:34:42.761943 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:42.761902    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:42.762455 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:42.761965    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:42.774555 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:42.774528    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:34:43.198557 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:34:43.198531    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:35:16.039052 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:16.039012    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"]
Jul 10 16:35:17.021086 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:17.021053    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"]
Jul 10 16:35:31.059856 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.059793    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v" podUID="362a595f-3f9c-486a-9267-c98f7c76db27" containerName="main" containerID="cri-o://843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e" gracePeriod=45
Jul 10 16:35:31.317596 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.317519    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:35:31.372328 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.372292    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362a595f-3f9c-486a-9267-c98f7c76db27-tls-certs\") pod \"362a595f-3f9c-486a-9267-c98f7c76db27\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") "
Jul 10 16:35:31.372572 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.372370    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-tmp-dir\") pod \"362a595f-3f9c-486a-9267-c98f7c76db27\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") "
Jul 10 16:35:31.372572 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.372468    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-dshm\") pod \"362a595f-3f9c-486a-9267-c98f7c76db27\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") "
Jul 10 16:35:31.372572 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.372521    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fbmc\" (UniqueName: \"kubernetes.io/projected/362a595f-3f9c-486a-9267-c98f7c76db27-kube-api-access-7fbmc\") pod \"362a595f-3f9c-486a-9267-c98f7c76db27\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") "
Jul 10 16:35:31.372753 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.372605    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-home\") pod \"362a595f-3f9c-486a-9267-c98f7c76db27\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") "
Jul 10 16:35:31.372753 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.372640    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "362a595f-3f9c-486a-9267-c98f7c76db27" (UID: "362a595f-3f9c-486a-9267-c98f7c76db27"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:35:31.372753 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.372647    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-model-cache\") pod \"362a595f-3f9c-486a-9267-c98f7c76db27\" (UID: \"362a595f-3f9c-486a-9267-c98f7c76db27\") "
Jul 10 16:35:31.372909 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.372886    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-model-cache" (OuterVolumeSpecName: "model-cache") pod "362a595f-3f9c-486a-9267-c98f7c76db27" (UID: "362a595f-3f9c-486a-9267-c98f7c76db27"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:35:31.372963 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.372901    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-home" (OuterVolumeSpecName: "home") pod "362a595f-3f9c-486a-9267-c98f7c76db27" (UID: "362a595f-3f9c-486a-9267-c98f7c76db27"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:35:31.373083 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.373063    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-home\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:31.373140 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.373089    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-model-cache\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:31.373140 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.373119    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-tmp-dir\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:31.374740 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.374711    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-dshm" (OuterVolumeSpecName: "dshm") pod "362a595f-3f9c-486a-9267-c98f7c76db27" (UID: "362a595f-3f9c-486a-9267-c98f7c76db27"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:35:31.374876 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.374859    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/362a595f-3f9c-486a-9267-c98f7c76db27-kube-api-access-7fbmc" (OuterVolumeSpecName: "kube-api-access-7fbmc") pod "362a595f-3f9c-486a-9267-c98f7c76db27" (UID: "362a595f-3f9c-486a-9267-c98f7c76db27"). InnerVolumeSpecName "kube-api-access-7fbmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:35:31.374876 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.374865    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/362a595f-3f9c-486a-9267-c98f7c76db27-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "362a595f-3f9c-486a-9267-c98f7c76db27" (UID: "362a595f-3f9c-486a-9267-c98f7c76db27"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 16:35:31.423882 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.423850    2567 generic.go:356] "Generic (PLEG): container finished" podID="362a595f-3f9c-486a-9267-c98f7c76db27" containerID="843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e" exitCode=0
Jul 10 16:35:31.424151 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.423891    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"
Jul 10 16:35:31.424151 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.423902    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v" event={"ID":"362a595f-3f9c-486a-9267-c98f7c76db27","Type":"ContainerDied","Data":"843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e"}
Jul 10 16:35:31.424151 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.423935    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v" event={"ID":"362a595f-3f9c-486a-9267-c98f7c76db27","Type":"ContainerDied","Data":"8b3d08cb9eebb27bee5526901d77789e56c4b17caedfa09861e0dbe2d4786244"}
Jul 10 16:35:31.424151 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.423951    2567 scope.go:117] "RemoveContainer" containerID="843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e"
Jul 10 16:35:31.433011 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.432989    2567 scope.go:117] "RemoveContainer" containerID="843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e"
Jul 10 16:35:31.433286 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:35:31.433271    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e\": container with ID starting with 843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e not found: ID does not exist" containerID="843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e"
Jul 10 16:35:31.433344 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.433296    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e"} err="failed to get container status \"843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e\": rpc error: code = NotFound desc = could not find container \"843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e\": container with ID starting with 843122beb3be9318de319411dc5fd13559ff09fe9fa9ebc10e5b12bd77d8c18e not found: ID does not exist"
Jul 10 16:35:31.452367 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.452334    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"]
Jul 10 16:35:31.458293 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.458263    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c32d0d/sub-test-model-c32d0d-kserve-6d86579c87-vdk5v"]
Jul 10 16:35:31.474117 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.474084    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362a595f-3f9c-486a-9267-c98f7c76db27-dshm\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:31.474117 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.474112    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fbmc\" (UniqueName: \"kubernetes.io/projected/362a595f-3f9c-486a-9267-c98f7c76db27-kube-api-access-7fbmc\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:31.474335 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:31.474125    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362a595f-3f9c-486a-9267-c98f7c76db27-tls-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:32.042177 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.042111    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b" podUID="0e53ae09-78f1-430f-97e3-f06261334817" containerName="main" containerID="cri-o://1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e" gracePeriod=45
Jul 10 16:35:32.308825 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.308794    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:35:32.382767 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.382715    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-home\") pod \"0e53ae09-78f1-430f-97e3-f06261334817\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") "
Jul 10 16:35:32.382968 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.382781    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-tmp-dir\") pod \"0e53ae09-78f1-430f-97e3-f06261334817\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") "
Jul 10 16:35:32.382968 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.382811    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-dshm\") pod \"0e53ae09-78f1-430f-97e3-f06261334817\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") "
Jul 10 16:35:32.382968 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.382885    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-model-cache\") pod \"0e53ae09-78f1-430f-97e3-f06261334817\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") "
Jul 10 16:35:32.382968 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.382952    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb9rk\" (UniqueName: \"kubernetes.io/projected/0e53ae09-78f1-430f-97e3-f06261334817-kube-api-access-jb9rk\") pod \"0e53ae09-78f1-430f-97e3-f06261334817\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") "
Jul 10 16:35:32.383194 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.382987    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e53ae09-78f1-430f-97e3-f06261334817-tls-certs\") pod \"0e53ae09-78f1-430f-97e3-f06261334817\" (UID: \"0e53ae09-78f1-430f-97e3-f06261334817\") "
Jul 10 16:35:32.383194 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.383093    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-home" (OuterVolumeSpecName: "home") pod "0e53ae09-78f1-430f-97e3-f06261334817" (UID: "0e53ae09-78f1-430f-97e3-f06261334817"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:35:32.383194 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.383102    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0e53ae09-78f1-430f-97e3-f06261334817" (UID: "0e53ae09-78f1-430f-97e3-f06261334817"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:35:32.383333 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.383269    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-home\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:32.383333 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.383287    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-tmp-dir\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:32.383539 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.383516    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-model-cache" (OuterVolumeSpecName: "model-cache") pod "0e53ae09-78f1-430f-97e3-f06261334817" (UID: "0e53ae09-78f1-430f-97e3-f06261334817"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:35:32.385162 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.385131    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-dshm" (OuterVolumeSpecName: "dshm") pod "0e53ae09-78f1-430f-97e3-f06261334817" (UID: "0e53ae09-78f1-430f-97e3-f06261334817"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:35:32.385270 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.385206    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e53ae09-78f1-430f-97e3-f06261334817-kube-api-access-jb9rk" (OuterVolumeSpecName: "kube-api-access-jb9rk") pod "0e53ae09-78f1-430f-97e3-f06261334817" (UID: "0e53ae09-78f1-430f-97e3-f06261334817"). InnerVolumeSpecName "kube-api-access-jb9rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:35:32.385490 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.385469    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e53ae09-78f1-430f-97e3-f06261334817-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0e53ae09-78f1-430f-97e3-f06261334817" (UID: "0e53ae09-78f1-430f-97e3-f06261334817"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 16:35:32.430124 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.430085    2567 generic.go:356] "Generic (PLEG): container finished" podID="0e53ae09-78f1-430f-97e3-f06261334817" containerID="1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e" exitCode=0
Jul 10 16:35:32.430124 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.430124    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"
Jul 10 16:35:32.430387 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.430179    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b" event={"ID":"0e53ae09-78f1-430f-97e3-f06261334817","Type":"ContainerDied","Data":"1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e"}
Jul 10 16:35:32.430387 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.430220    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b" event={"ID":"0e53ae09-78f1-430f-97e3-f06261334817","Type":"ContainerDied","Data":"2c2cce20ff8e4d9032042a1b0f44ab54f1f9a82d37b0c196284f1fc35dd29a2a"}
Jul 10 16:35:32.430387 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.430241    2567 scope.go:117] "RemoveContainer" containerID="1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e"
Jul 10 16:35:32.439281 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.439262    2567 scope.go:117] "RemoveContainer" containerID="1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e"
Jul 10 16:35:32.439626 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:35:32.439605    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e\": container with ID starting with 1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e not found: ID does not exist" containerID="1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e"
Jul 10 16:35:32.439691 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.439639    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e"} err="failed to get container status \"1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e\": rpc error: code = NotFound desc = could not find container \"1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e\": container with ID starting with 1dd9d237a510109d93bd91f7ae511a1671e11b332a27d720790858fe5adfd82e not found: ID does not exist"
Jul 10 16:35:32.450815 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.450776    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"]
Jul 10 16:35:32.454241 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.454213    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/sub-test-model-466b1a-kserve-75554b95f-tll7b"]
Jul 10 16:35:32.484240 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.483990    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-dshm\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:32.484240 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.484026    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e53ae09-78f1-430f-97e3-f06261334817-model-cache\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:32.484240 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.484048    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb9rk\" (UniqueName: \"kubernetes.io/projected/0e53ae09-78f1-430f-97e3-f06261334817-kube-api-access-jb9rk\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:32.484240 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:32.484064    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e53ae09-78f1-430f-97e3-f06261334817-tls-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:35:33.405529 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:33.405499    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e53ae09-78f1-430f-97e3-f06261334817" path="/var/lib/kubelet/pods/0e53ae09-78f1-430f-97e3-f06261334817/volumes"
Jul 10 16:35:33.405953 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:33.405896    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="362a595f-3f9c-486a-9267-c98f7c76db27" path="/var/lib/kubelet/pods/362a595f-3f9c-486a-9267-c98f7c76db27/volumes"
Jul 10 16:35:47.441417 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.441316    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"]
Jul 10 16:35:47.441868 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.441696    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="362a595f-3f9c-486a-9267-c98f7c76db27" containerName="main"
Jul 10 16:35:47.441868 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.441708    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="362a595f-3f9c-486a-9267-c98f7c76db27" containerName="main"
Jul 10 16:35:47.441868 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.441721    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e53ae09-78f1-430f-97e3-f06261334817" containerName="main"
Jul 10 16:35:47.441868 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.441726    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e53ae09-78f1-430f-97e3-f06261334817" containerName="main"
Jul 10 16:35:47.441868 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.441810    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e53ae09-78f1-430f-97e3-f06261334817" containerName="main"
Jul 10 16:35:47.441868 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.441817    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="362a595f-3f9c-486a-9267-c98f7c76db27" containerName="main"
Jul 10 16:35:47.446055 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.446038    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.449787 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.449751    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"default-dockercfg-k7jrd\""
Jul 10 16:35:47.449931 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.449809    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"kube-root-ca.crt\""
Jul 10 16:35:47.450004 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.449927    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"openshift-service-ca.crt\""
Jul 10 16:35:47.450072 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.450031    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-466b1a\"/\"rate-test-model-466b1a-kserve-self-signed-certs\""
Jul 10 16:35:47.453134 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.453110    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"]
Jul 10 16:35:47.529201 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.529164    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-home\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.529385 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.529240    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9344599-d2a1-4424-9e22-55b05dc92e98-tls-certs\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.529385 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.529340    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-model-cache\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.529385 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.529370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4vd9\" (UniqueName: \"kubernetes.io/projected/a9344599-d2a1-4424-9e22-55b05dc92e98-kube-api-access-m4vd9\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.529548 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.529491    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-dshm\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.529548 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.529542    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-tmp-dir\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.630756 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.630697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-model-cache\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.630756 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.630757    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4vd9\" (UniqueName: \"kubernetes.io/projected/a9344599-d2a1-4424-9e22-55b05dc92e98-kube-api-access-m4vd9\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.630985 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.630813    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-dshm\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.630985 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.630852    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-tmp-dir\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.630985 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.630888    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-home\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.630985 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.630931    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9344599-d2a1-4424-9e22-55b05dc92e98-tls-certs\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.631215 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.631180    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-model-cache\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.631360 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.631252    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-tmp-dir\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.631360 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.631314    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-home\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.633366 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.633341    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-dshm\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.633535 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.633517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9344599-d2a1-4424-9e22-55b05dc92e98-tls-certs\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.640009 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.639986    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4vd9\" (UniqueName: \"kubernetes.io/projected/a9344599-d2a1-4424-9e22-55b05dc92e98-kube-api-access-m4vd9\") pod \"rate-test-model-466b1a-kserve-74c4ff484c-xqlg9\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") " pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.757399 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.757363    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:47.903847 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:47.903811    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"]
Jul 10 16:35:47.906958 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:35:47.906929    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9344599_d2a1_4424_9e22_55b05dc92e98.slice/crio-f6492fdccdf1d923b53a9ea5b1891e8bef60de7e32dca067c2d78f163ca80dbe WatchSource:0}: Error finding container f6492fdccdf1d923b53a9ea5b1891e8bef60de7e32dca067c2d78f163ca80dbe: Status 404 returned error can't find the container with id f6492fdccdf1d923b53a9ea5b1891e8bef60de7e32dca067c2d78f163ca80dbe
Jul 10 16:35:48.508050 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:48.508005    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9" event={"ID":"a9344599-d2a1-4424-9e22-55b05dc92e98","Type":"ContainerStarted","Data":"03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813"}
Jul 10 16:35:48.508478 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:48.508064    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9" event={"ID":"a9344599-d2a1-4424-9e22-55b05dc92e98","Type":"ContainerStarted","Data":"f6492fdccdf1d923b53a9ea5b1891e8bef60de7e32dca067c2d78f163ca80dbe"}
Jul 10 16:35:48.529308 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:48.529254    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9" podStartSLOduration=1.529234873 podStartE2EDuration="1.529234873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:35:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:35:48.527228228 +0000 UTC m=+2435.670762070" watchObservedRunningTime="2026-07-10 16:35:48.529234873 +0000 UTC m=+2435.672768711"
Jul 10 16:35:57.757763 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:57.757722    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:57.757763 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:57.757774    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:57.770837 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:57.770802    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:35:58.567398 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:35:58.567360    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:36:39.427562 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:39.427524    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"]
Jul 10 16:36:54.449031 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.448964    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9" podUID="a9344599-d2a1-4424-9e22-55b05dc92e98" containerName="main" containerID="cri-o://03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813" gracePeriod=45
Jul 10 16:36:54.704074 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.703983    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:36:54.755120 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755072    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-model-cache\") pod \"a9344599-d2a1-4424-9e22-55b05dc92e98\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") "
Jul 10 16:36:54.755313 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755153    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4vd9\" (UniqueName: \"kubernetes.io/projected/a9344599-d2a1-4424-9e22-55b05dc92e98-kube-api-access-m4vd9\") pod \"a9344599-d2a1-4424-9e22-55b05dc92e98\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") "
Jul 10 16:36:54.755313 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755176    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-dshm\") pod \"a9344599-d2a1-4424-9e22-55b05dc92e98\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") "
Jul 10 16:36:54.755313 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755193    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-tmp-dir\") pod \"a9344599-d2a1-4424-9e22-55b05dc92e98\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") "
Jul 10 16:36:54.755313 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755234    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9344599-d2a1-4424-9e22-55b05dc92e98-tls-certs\") pod \"a9344599-d2a1-4424-9e22-55b05dc92e98\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") "
Jul 10 16:36:54.755590 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755320    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-home\") pod \"a9344599-d2a1-4424-9e22-55b05dc92e98\" (UID: \"a9344599-d2a1-4424-9e22-55b05dc92e98\") "
Jul 10 16:36:54.755590 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755433    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-model-cache" (OuterVolumeSpecName: "model-cache") pod "a9344599-d2a1-4424-9e22-55b05dc92e98" (UID: "a9344599-d2a1-4424-9e22-55b05dc92e98"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:36:54.755590 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755558    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a9344599-d2a1-4424-9e22-55b05dc92e98" (UID: "a9344599-d2a1-4424-9e22-55b05dc92e98"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:36:54.755743 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755649    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-home" (OuterVolumeSpecName: "home") pod "a9344599-d2a1-4424-9e22-55b05dc92e98" (UID: "a9344599-d2a1-4424-9e22-55b05dc92e98"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:36:54.755805 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755762    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-home\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:36:54.755805 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755778    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-model-cache\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:36:54.755805 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.755792    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-tmp-dir\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:36:54.757812 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.757777    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-dshm" (OuterVolumeSpecName: "dshm") pod "a9344599-d2a1-4424-9e22-55b05dc92e98" (UID: "a9344599-d2a1-4424-9e22-55b05dc92e98"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:36:54.757812 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.757793    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9344599-d2a1-4424-9e22-55b05dc92e98-kube-api-access-m4vd9" (OuterVolumeSpecName: "kube-api-access-m4vd9") pod "a9344599-d2a1-4424-9e22-55b05dc92e98" (UID: "a9344599-d2a1-4424-9e22-55b05dc92e98"). InnerVolumeSpecName "kube-api-access-m4vd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:36:54.758026 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.757972    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9344599-d2a1-4424-9e22-55b05dc92e98-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a9344599-d2a1-4424-9e22-55b05dc92e98" (UID: "a9344599-d2a1-4424-9e22-55b05dc92e98"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 16:36:54.821535 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.821496    2567 generic.go:356] "Generic (PLEG): container finished" podID="a9344599-d2a1-4424-9e22-55b05dc92e98" containerID="03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813" exitCode=0
Jul 10 16:36:54.821701 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.821540    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"
Jul 10 16:36:54.821701 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.821578    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9" event={"ID":"a9344599-d2a1-4424-9e22-55b05dc92e98","Type":"ContainerDied","Data":"03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813"}
Jul 10 16:36:54.821701 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.821624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9" event={"ID":"a9344599-d2a1-4424-9e22-55b05dc92e98","Type":"ContainerDied","Data":"f6492fdccdf1d923b53a9ea5b1891e8bef60de7e32dca067c2d78f163ca80dbe"}
Jul 10 16:36:54.821701 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.821644    2567 scope.go:117] "RemoveContainer" containerID="03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813"
Jul 10 16:36:54.830952 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.830931    2567 scope.go:117] "RemoveContainer" containerID="03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813"
Jul 10 16:36:54.831227 ip-10-0-135-156 kubenswrapper[2567]: E0710 16:36:54.831210    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813\": container with ID starting with 03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813 not found: ID does not exist" containerID="03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813"
Jul 10 16:36:54.831271 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.831236    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813"} err="failed to get container status \"03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813\": rpc error: code = NotFound desc = could not find container \"03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813\": container with ID starting with 03f22b64141871ea3f9db4922eb0c985682eb213b5932ceb545c781e37f4e813 not found: ID does not exist"
Jul 10 16:36:54.844136 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.844110    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"]
Jul 10 16:36:54.848276 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.848249    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-466b1a/rate-test-model-466b1a-kserve-74c4ff484c-xqlg9"]
Jul 10 16:36:54.856911 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.856894    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9344599-d2a1-4424-9e22-55b05dc92e98-tls-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:36:54.856982 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.856914    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4vd9\" (UniqueName: \"kubernetes.io/projected/a9344599-d2a1-4424-9e22-55b05dc92e98-kube-api-access-m4vd9\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:36:54.856982 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:54.856925    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9344599-d2a1-4424-9e22-55b05dc92e98-dshm\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:36:55.405889 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:36:55.405856    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9344599-d2a1-4424-9e22-55b05dc92e98" path="/var/lib/kubelet/pods/a9344599-d2a1-4424-9e22-55b05dc92e98/volumes"
Jul 10 16:37:12.776508 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:12.776426    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"]
Jul 10 16:37:12.776957 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:12.776674    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" podUID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerName="istio-proxy" containerID="cri-o://0b5841c7b52ce1bbb005f2cff7bfdc1c039f5968fafa8a09b3839cbfaa2464fe" gracePeriod=30
Jul 10 16:37:17.931779 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:17.931740    2567 generic.go:356] "Generic (PLEG): container finished" podID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerID="0b5841c7b52ce1bbb005f2cff7bfdc1c039f5968fafa8a09b3839cbfaa2464fe" exitCode=0
Jul 10 16:37:17.932188 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:17.931805    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" event={"ID":"449c4b13-9e17-4ca4-b83f-721aaed4c704","Type":"ContainerDied","Data":"0b5841c7b52ce1bbb005f2cff7bfdc1c039f5968fafa8a09b3839cbfaa2464fe"}
Jul 10 16:37:17.932188 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:17.931873    2567 scope.go:117] "RemoveContainer" containerID="a6511cfbfda5fa9b10d0a1b904df08ad1ca45456971d257308da147ce6d468a1"
Jul 10 16:37:18.044789 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.044714    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:37:18.180002 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.179961    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-token\") pod \"449c4b13-9e17-4ca4-b83f-721aaed4c704\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") "
Jul 10 16:37:18.180182 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180050    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-envoy\") pod \"449c4b13-9e17-4ca4-b83f-721aaed4c704\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") "
Jul 10 16:37:18.180182 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180104    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-data\") pod \"449c4b13-9e17-4ca4-b83f-721aaed4c704\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") "
Jul 10 16:37:18.180182 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180137    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/449c4b13-9e17-4ca4-b83f-721aaed4c704-istiod-ca-cert\") pod \"449c4b13-9e17-4ca4-b83f-721aaed4c704\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") "
Jul 10 16:37:18.180182 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180165    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-credential-socket\") pod \"449c4b13-9e17-4ca4-b83f-721aaed4c704\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") "
Jul 10 16:37:18.180352 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180247    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-certs\") pod \"449c4b13-9e17-4ca4-b83f-721aaed4c704\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") "
Jul 10 16:37:18.180352 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180285    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfkh2\" (UniqueName: \"kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-kube-api-access-xfkh2\") pod \"449c4b13-9e17-4ca4-b83f-721aaed4c704\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") "
Jul 10 16:37:18.180352 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180308    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-socket\") pod \"449c4b13-9e17-4ca4-b83f-721aaed4c704\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") "
Jul 10 16:37:18.180352 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180337    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-podinfo\") pod \"449c4b13-9e17-4ca4-b83f-721aaed4c704\" (UID: \"449c4b13-9e17-4ca4-b83f-721aaed4c704\") "
Jul 10 16:37:18.180747 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180640    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/449c4b13-9e17-4ca4-b83f-721aaed4c704-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "449c4b13-9e17-4ca4-b83f-721aaed4c704" (UID: "449c4b13-9e17-4ca4-b83f-721aaed4c704"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 16:37:18.180747 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180640    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "449c4b13-9e17-4ca4-b83f-721aaed4c704" (UID: "449c4b13-9e17-4ca4-b83f-721aaed4c704"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:37:18.180966 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180753    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "449c4b13-9e17-4ca4-b83f-721aaed4c704" (UID: "449c4b13-9e17-4ca4-b83f-721aaed4c704"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:37:18.180966 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180769    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "449c4b13-9e17-4ca4-b83f-721aaed4c704" (UID: "449c4b13-9e17-4ca4-b83f-721aaed4c704"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:37:18.180966 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.180955    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-data" (OuterVolumeSpecName: "istio-data") pod "449c4b13-9e17-4ca4-b83f-721aaed4c704" (UID: "449c4b13-9e17-4ca4-b83f-721aaed4c704"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:37:18.182585 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.182564    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "449c4b13-9e17-4ca4-b83f-721aaed4c704" (UID: "449c4b13-9e17-4ca4-b83f-721aaed4c704"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 16:37:18.182692 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.182617    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-token" (OuterVolumeSpecName: "istio-token") pod "449c4b13-9e17-4ca4-b83f-721aaed4c704" (UID: "449c4b13-9e17-4ca4-b83f-721aaed4c704"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:37:18.182756 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.182734    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-kube-api-access-xfkh2" (OuterVolumeSpecName: "kube-api-access-xfkh2") pod "449c4b13-9e17-4ca4-b83f-721aaed4c704" (UID: "449c4b13-9e17-4ca4-b83f-721aaed4c704"). InnerVolumeSpecName "kube-api-access-xfkh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 16:37:18.182837 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.182822    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "449c4b13-9e17-4ca4-b83f-721aaed4c704" (UID: "449c4b13-9e17-4ca4-b83f-721aaed4c704"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 16:37:18.282109 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.282069    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-certs\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:37:18.282109 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.282100    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfkh2\" (UniqueName: \"kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-kube-api-access-xfkh2\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:37:18.282109 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.282110    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-workload-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:37:18.282109 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.282119    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-podinfo\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:37:18.282369 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.282128    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-token\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:37:18.282369 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.282136    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-envoy\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:37:18.282369 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.282142    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-istio-data\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:37:18.282369 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.282152    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/449c4b13-9e17-4ca4-b83f-721aaed4c704-istiod-ca-cert\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:37:18.282369 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.282159    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/449c4b13-9e17-4ca4-b83f-721aaed4c704-credential-socket\") on node \"ip-10-0-135-156.ec2.internal\" DevicePath \"\""
Jul 10 16:37:18.645785 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.645740    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-svpd7_c2831a0f-80d6-4601-8c09-e4454c84dd25/manager/0.log"
Jul 10 16:37:18.887771 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.887740    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-n7b4v_3e98b2d3-ce25-46d7-b77d-46377f5dd677/server/0.log"
Jul 10 16:37:18.937757 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.937660    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"
Jul 10 16:37:18.937757 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.937684    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz" event={"ID":"449c4b13-9e17-4ca4-b83f-721aaed4c704","Type":"ContainerDied","Data":"a33eee45fbe969d78b6299ac15f63f319b03fc0e21f7e36af05ffc9ddc8583b7"}
Jul 10 16:37:18.937757 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.937752    2567 scope.go:117] "RemoveContainer" containerID="0b5841c7b52ce1bbb005f2cff7bfdc1c039f5968fafa8a09b3839cbfaa2464fe"
Jul 10 16:37:18.961248 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.961217    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"]
Jul 10 16:37:18.964318 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:18.964292    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-466b1a-openshift-default-59c79b5759-rzjsz"]
Jul 10 16:37:19.122060 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:19.122028    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm_fa3173d1-73f1-4797-8b79-b80137b2ce7a/copy-manifests/0.log"
Jul 10 16:37:19.131208 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:19.131182    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm_fa3173d1-73f1-4797-8b79-b80137b2ce7a/manager/0.log"
Jul 10 16:37:19.405488 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:19.405458    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="449c4b13-9e17-4ca4-b83f-721aaed4c704" path="/var/lib/kubelet/pods/449c4b13-9e17-4ca4-b83f-721aaed4c704/volumes"
Jul 10 16:37:21.307298 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:21.307255    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-22bgx_a0f53ac0-4ac4-4dd9-b501-c7f9de14cde2/manager/0.log"
Jul 10 16:37:22.039398 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:22.039367    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-njgsl_8c490cbd-0f85-4557-9921-b674a2c3e808/manager/0.log"
Jul 10 16:37:22.389174 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:22.389070    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qv8fg_b1a65958-a086-4f31-b544-8aeebcc054e3/istio-proxy/0.log"
Jul 10 16:37:23.225871 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:23.225835    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59786c5dc6-qhtdf_ef2cf27d-6566-4821-9282-c222c9b35c50/router/0.log"
Jul 10 16:37:23.686856 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:23.686776    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5trlm_6a4dbbf4-0334-4eaf-987a-f01a0a28712d/main/0.log"
Jul 10 16:37:23.912838 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:23.912801    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmtglf_7234765d-4dc7-4791-82da-2389c8ee16f7/main/0.log"
Jul 10 16:37:24.148893 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:24.148843    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-hc7np_cd6528d4-8cfe-4fbd-81b7-419ae29e77a3/main/0.log"
Jul 10 16:37:31.930281 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:31.930248    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-l5nkc_16d3e229-e008-438f-8f5a-13d9364e5b7b/global-pull-secret-syncer/0.log"
Jul 10 16:37:32.076144 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:32.076107    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hw282_9133bd56-dbfb-4b6a-8b50-d96c637055a3/konnectivity-agent/0.log"
Jul 10 16:37:32.144018 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:32.143985    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-156.ec2.internal_6a91c7d5b3d2028299be24baf3606bd6/haproxy/0.log"
Jul 10 16:37:37.790968 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:37.790930    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-22bgx_a0f53ac0-4ac4-4dd9-b501-c7f9de14cde2/manager/0.log"
Jul 10 16:37:38.103300 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:38.103193    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-njgsl_8c490cbd-0f85-4557-9921-b674a2c3e808/manager/0.log"
Jul 10 16:37:40.039178 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:40.039138    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-8lsrg_faced6ae-ed28-45dd-b9c2-7d86892254f4/monitoring-plugin/0.log"
Jul 10 16:37:40.255885 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:40.255851    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cj2vz_e68dd708-b1d0-48b8-8926-7824da62145c/node-exporter/0.log"
Jul 10 16:37:40.309675 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:40.309570    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cj2vz_e68dd708-b1d0-48b8-8926-7824da62145c/kube-rbac-proxy/0.log"
Jul 10 16:37:40.335635 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:40.335603    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cj2vz_e68dd708-b1d0-48b8-8926-7824da62145c/init-textfile/0.log"
Jul 10 16:37:40.640318 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:40.640217    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e0e15cb7-dbbd-4d21-9625-4a4cf716c296/prometheus/0.log"
Jul 10 16:37:40.663716 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:40.663689    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e0e15cb7-dbbd-4d21-9625-4a4cf716c296/config-reloader/0.log"
Jul 10 16:37:40.686335 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:40.686268    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e0e15cb7-dbbd-4d21-9625-4a4cf716c296/thanos-sidecar/0.log"
Jul 10 16:37:40.707234 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:40.707211    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e0e15cb7-dbbd-4d21-9625-4a4cf716c296/kube-rbac-proxy-web/0.log"
Jul 10 16:37:40.729005 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:40.728964    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e0e15cb7-dbbd-4d21-9625-4a4cf716c296/kube-rbac-proxy/0.log"
Jul 10 16:37:40.757430 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:40.757373    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e0e15cb7-dbbd-4d21-9625-4a4cf716c296/kube-rbac-proxy-thanos/0.log"
Jul 10 16:37:40.781281 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:40.781253    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e0e15cb7-dbbd-4d21-9625-4a4cf716c296/init-config-reloader/0.log"
Jul 10 16:37:41.922385 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.922353    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"]
Jul 10 16:37:41.922778 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.922697    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerName="istio-proxy"
Jul 10 16:37:41.922778 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.922708    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerName="istio-proxy"
Jul 10 16:37:41.922778 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.922721    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9344599-d2a1-4424-9e22-55b05dc92e98" containerName="main"
Jul 10 16:37:41.922778 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.922726    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9344599-d2a1-4424-9e22-55b05dc92e98" containerName="main"
Jul 10 16:37:41.922911 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.922791    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerName="istio-proxy"
Jul 10 16:37:41.922911 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.922799    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="449c4b13-9e17-4ca4-b83f-721aaed4c704" containerName="istio-proxy"
Jul 10 16:37:41.922911 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.922806    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9344599-d2a1-4424-9e22-55b05dc92e98" containerName="main"
Jul 10 16:37:41.927250 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.927231    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:41.929923 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.929899    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wrzbj\"/\"kube-root-ca.crt\""
Jul 10 16:37:41.930137 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.930064    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wrzbj\"/\"default-dockercfg-bbxt2\""
Jul 10 16:37:41.931034 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.931015    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wrzbj\"/\"openshift-service-ca.crt\""
Jul 10 16:37:41.935692 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:41.935668    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"]
Jul 10 16:37:42.007976 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.007930    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-proc\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.008185 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.008054    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p69nx\" (UniqueName: \"kubernetes.io/projected/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-kube-api-access-p69nx\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.008185 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.008111    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-podres\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.008185 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.008151    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-lib-modules\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.008185 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.008181    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-sys\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.109630 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.109590    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p69nx\" (UniqueName: \"kubernetes.io/projected/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-kube-api-access-p69nx\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.109630 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.109635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-podres\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.109903 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.109675    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-lib-modules\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.109903 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.109701    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-sys\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.109903 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.109734    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-proc\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.109903 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.109823    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-proc\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.109903 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.109822    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-sys\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.109903 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.109861    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-podres\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.109903 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.109865    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-lib-modules\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.119366 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.119340    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p69nx\" (UniqueName: \"kubernetes.io/projected/3d8c4da4-bf93-4b26-9c20-829ce58e7aa4-kube-api-access-p69nx\") pod \"perf-node-gather-daemonset-msqtl\" (UID: \"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4\") " pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.238593 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.238559    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:42.277725 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.277692    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-bdtkv_ebf7138d-5fef-4905-9e1f-64c011efc16d/networking-console-plugin/0.log"
Jul 10 16:37:42.383003 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:42.382977    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"]
Jul 10 16:37:42.385178 ip-10-0-135-156 kubenswrapper[2567]: W0710 16:37:42.385138    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3d8c4da4_bf93_4b26_9c20_829ce58e7aa4.slice/crio-91229fe364962bfa03b7d2f6cb9217ba7d86ffdb4762423191b70769c470d10d WatchSource:0}: Error finding container 91229fe364962bfa03b7d2f6cb9217ba7d86ffdb4762423191b70769c470d10d: Status 404 returned error can't find the container with id 91229fe364962bfa03b7d2f6cb9217ba7d86ffdb4762423191b70769c470d10d
Jul 10 16:37:43.049564 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:43.049526    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl" event={"ID":"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4","Type":"ContainerStarted","Data":"b6e3813725e4fe2e1212dc1fed1045dbdcccdbba130b74a375889a7caf49f0bb"}
Jul 10 16:37:43.050010 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:43.049608    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl" event={"ID":"3d8c4da4-bf93-4b26-9c20-829ce58e7aa4","Type":"ContainerStarted","Data":"91229fe364962bfa03b7d2f6cb9217ba7d86ffdb4762423191b70769c470d10d"}
Jul 10 16:37:43.050010 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:43.049677    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:43.069255 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:43.069195    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl" podStartSLOduration=2.069176294 podStartE2EDuration="2.069176294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:37:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:37:43.06890141 +0000 UTC m=+2550.212435251" watchObservedRunningTime="2026-07-10 16:37:43.069176294 +0000 UTC m=+2550.212710134"
Jul 10 16:37:44.020061 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:44.020012    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-g9rb5_8ffb2acb-7443-471d-8755-3542c855072f/volume-data-source-validator/0.log"
Jul 10 16:37:45.148295 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:45.148256    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z8vr5_a6f5c915-40a8-4894-b83a-01985ca022fb/dns/0.log"
Jul 10 16:37:45.176957 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:45.176910    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z8vr5_a6f5c915-40a8-4894-b83a-01985ca022fb/kube-rbac-proxy/0.log"
Jul 10 16:37:45.302950 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:45.302920    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-htc8d_76d52889-8bfd-412c-b4fe-3943f4e074a0/dns-node-resolver/0.log"
Jul 10 16:37:45.873986 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:45.873949    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-84cd795589-bb48p_bd67177f-3b05-40fc-a5ad-7c23c53a9253/registry/0.log"
Jul 10 16:37:45.927423 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:45.927387    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kdsx4_9fc05a30-9841-4627-a43e-5591964cce62/node-ca/0.log"
Jul 10 16:37:46.926782 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:46.926737    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qv8fg_b1a65958-a086-4f31-b544-8aeebcc054e3/istio-proxy/0.log"
Jul 10 16:37:47.240049 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:47.240003    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59786c5dc6-qhtdf_ef2cf27d-6566-4821-9282-c222c9b35c50/router/0.log"
Jul 10 16:37:47.981813 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:47.981771    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dch4q_e21c179a-863d-43f0-9982-f10559073d45/serve-healthcheck-canary/0.log"
Jul 10 16:37:48.516593 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:48.516559    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-gphds_4415dbe9-f215-4e32-b06a-f3fe0affea2c/insights-operator/0.log"
Jul 10 16:37:48.517383 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:48.517355    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-gphds_4415dbe9-f215-4e32-b06a-f3fe0affea2c/insights-operator/1.log"
Jul 10 16:37:48.696315 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:48.696289    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sjb7f_194db3c8-57e0-4bec-bef7-c2ef3ca7fce7/kube-rbac-proxy/0.log"
Jul 10 16:37:48.725866 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:48.725840    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sjb7f_194db3c8-57e0-4bec-bef7-c2ef3ca7fce7/exporter/0.log"
Jul 10 16:37:48.750650 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:48.750605    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sjb7f_194db3c8-57e0-4bec-bef7-c2ef3ca7fce7/extractor/0.log"
Jul 10 16:37:49.064927 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:49.064900    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wrzbj/perf-node-gather-daemonset-msqtl"
Jul 10 16:37:51.434347 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:51.434311    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-svpd7_c2831a0f-80d6-4601-8c09-e4454c84dd25/manager/0.log"
Jul 10 16:37:51.557901 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:51.557807    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-n7b4v_3e98b2d3-ce25-46d7-b77d-46377f5dd677/server/0.log"
Jul 10 16:37:51.657805 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:51.657776    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm_fa3173d1-73f1-4797-8b79-b80137b2ce7a/manager/0.log"
Jul 10 16:37:51.679921 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:51.679897    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c56c8c4bf-jv2zm_fa3173d1-73f1-4797-8b79-b80137b2ce7a/copy-manifests/0.log"
Jul 10 16:37:52.816473 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:52.816441    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qv8fg_b1a65958-a086-4f31-b544-8aeebcc054e3/istio-proxy/0.log"
Jul 10 16:37:53.059948 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:53.059834    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59786c5dc6-qhtdf_ef2cf27d-6566-4821-9282-c222c9b35c50/router/0.log"
Jul 10 16:37:53.092317 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:53.092267    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66568bbc8f-bpgkk_d3f16d5e-b928-44fc-aa0d-b7cf3779a615/manager/0.log"
Jul 10 16:37:54.274719 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:54.274685    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qv8fg_b1a65958-a086-4f31-b544-8aeebcc054e3/istio-proxy/0.log"
Jul 10 16:37:54.545331 ip-10-0-135-156 kubenswrapper[2567]: I0710 16:37:54.545232    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59786c5dc6-qhtdf_ef2cf27d-6566-4821-9282-c222c9b35c50/router/0.log"