Jul 15 10:45:22.104542 ip-10-0-134-178 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 10:45:22.417707 ip-10-0-134-178 kubenswrapper[2536]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 10:45:22.417707 ip-10-0-134-178 kubenswrapper[2536]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 10:45:22.417707 ip-10-0-134-178 kubenswrapper[2536]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 10:45:22.418339 ip-10-0-134-178 kubenswrapper[2536]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 10:45:22.418339 ip-10-0-134-178 kubenswrapper[2536]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 10:45:22.418339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.417779    2536 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 10:45:22.422736 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422720    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 10:45:22.422736 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422735    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 10:45:22.422736 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422739    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422743    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422745    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422748    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422750    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422754    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422757    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422759    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422762    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422764    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422767    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422770    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422772    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422775    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422777    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422780    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422782    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422785    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422787    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422789    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 10:45:22.422823 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422792    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422795    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422802    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422805    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422808    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422810    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422813    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422815    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422818    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422820    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422823    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422825    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422828    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422831    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422833    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422836    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422839    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422843    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422847    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 10:45:22.423417 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422850    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422853    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422857    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422861    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422864    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422867    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422869    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422872    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422875    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422877    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422880    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422882    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422885    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422887    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422890    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422893    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422895    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422898    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422900    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 10:45:22.423869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422903    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422906    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422909    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422911    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422914    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422916    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422918    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422922    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422924    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422927    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422930    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422932    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422934    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422937    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422940    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422942    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422944    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422947    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422950    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422952    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 10:45:22.424351 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422955    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422957    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422960    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422962    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422964    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422967    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422969    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422972    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422976    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422980    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422983    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422986    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422988    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422991    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422993    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422996    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.422999    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.423001    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 10:45:22.424836 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.423003    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425240    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425249    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425253    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425256    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425259    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425261    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425264    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425267    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425270    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425284    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425287    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425290    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425293    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425295    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425298    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425300    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425303    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425305    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425308    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 10:45:22.425323 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425310    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425313    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425315    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425318    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425320    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425322    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425325    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425327    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425330    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425333    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425335    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425338    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425341    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425343    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425346    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425348    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425350    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425353    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425355    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 10:45:22.425818 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425357    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425360    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425362    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425364    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425367    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425369    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425372    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425374    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425376    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425379    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425381    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425383    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425385    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425388    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425390    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425393    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425396    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425398    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425400    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 10:45:22.426318 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425404    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425407    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425410    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425412    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425414    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425417    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425420    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425422    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425424    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425427    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425430    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425432    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425435    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425437    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425441    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425444    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425447    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425450    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425452    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425455    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 10:45:22.426791 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425458    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425461    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425463    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425466    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425469    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425472    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425474    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425476    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425479    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425482    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425484    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425488    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425490    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425493    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425495    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425498    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425500    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425502    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425506    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 10:45:22.427286 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425509    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425512    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425515    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425862    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425867    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425870    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425873    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425876    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425879    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425882    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425885    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425888    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425891    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425894    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425897    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425899    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425902    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425906    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425909    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425911    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 10:45:22.427757 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425914    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425917    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425920    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425922    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425925    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425927    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425929    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425933    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425936    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425938    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425941    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425943    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425946    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425948    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425950    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425953    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425955    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425958    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 10:45:22.428234 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425961    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425965    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425968    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425971    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425973    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425976    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425979    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425981    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425984    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425988    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425990    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425993    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425996    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.425998    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426001    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426004    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426006    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426009    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426011    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426014    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 10:45:22.428679 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426016    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426019    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426021    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426024    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426026    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426028    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426031    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426033    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426035    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426037    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426040    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426042    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426046    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426049    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426052    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426055    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426059    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426061    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426081    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426097    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 10:45:22.429175 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426101    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426104    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426108    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426110    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426113    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426116    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426118    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426121    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426124    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426127    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426129    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426132    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426135    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426137    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426140    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426143    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426146    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426148    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426151    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426153    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 10:45:22.429683 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426156    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426158    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426161    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426163    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426245    2536 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426252    2536 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426259    2536 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426263    2536 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426268    2536 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426284    2536 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426290    2536 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426296    2536 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426299    2536 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426302    2536 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426305    2536 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426308    2536 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426311    2536 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426314    2536 flags.go:64] FLAG: --cgroup-root=""
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426317    2536 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426320    2536 flags.go:64] FLAG: --client-ca-file=""
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426322    2536 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426325    2536 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426329    2536 flags.go:64] FLAG: --cluster-domain=""
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426332    2536 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 10:45:22.430188 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426335    2536 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426338    2536 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426341    2536 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426345    2536 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426348    2536 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426351    2536 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426354    2536 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426357    2536 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426359    2536 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426362    2536 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426365    2536 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426368    2536 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426372    2536 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426375    2536 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426377    2536 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426380    2536 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426382    2536 flags.go:64] FLAG: --enable-server="true"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426385    2536 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426391    2536 flags.go:64] FLAG: --event-burst="100"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426394    2536 flags.go:64] FLAG: --event-qps="50"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426398    2536 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426400    2536 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426403    2536 flags.go:64] FLAG: --eviction-hard=""
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426407    2536 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 10:45:22.430785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426410    2536 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426413    2536 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426416    2536 flags.go:64] FLAG: --eviction-soft=""
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426418    2536 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426421    2536 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426424    2536 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426426    2536 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426429    2536 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426432    2536 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426434    2536 flags.go:64] FLAG: --feature-gates=""
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426438    2536 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426441    2536 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426443    2536 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426446    2536 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426449    2536 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426452    2536 flags.go:64] FLAG: --help="false"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426455    2536 flags.go:64] FLAG: --hostname-override="ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426458    2536 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426460    2536 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426463    2536 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426466    2536 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426469    2536 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426472    2536 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 10:45:22.431378 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426475    2536 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426481    2536 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426483    2536 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426486    2536 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426490    2536 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426493    2536 flags.go:64] FLAG: --kube-reserved=""
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426496    2536 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426499    2536 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426502    2536 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426504    2536 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426507    2536 flags.go:64] FLAG: --lock-file=""
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426510    2536 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426513    2536 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426515    2536 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426521    2536 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426524    2536 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426527    2536 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426529    2536 flags.go:64] FLAG: --logging-format="text"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426532    2536 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426535    2536 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426538    2536 flags.go:64] FLAG: --manifest-url=""
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426541    2536 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426545    2536 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426548    2536 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426551    2536 flags.go:64] FLAG: --max-pods="110"
Jul 15 10:45:22.431953 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426554    2536 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426557    2536 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426560    2536 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426562    2536 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426565    2536 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426568    2536 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426571    2536 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426583    2536 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426588    2536 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426591    2536 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426594    2536 flags.go:64] FLAG: --pod-cidr=""
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426597    2536 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426602    2536 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426605    2536 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426611    2536 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426614    2536 flags.go:64] FLAG: --port="10250"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426617    2536 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426620    2536 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f9015512958ebbb3"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426623    2536 flags.go:64] FLAG: --qos-reserved=""
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426626    2536 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426629    2536 flags.go:64] FLAG: --register-node="true"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426631    2536 flags.go:64] FLAG: --register-with-taints=""
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426635    2536 flags.go:64] FLAG: --registry-burst="10"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426637    2536 flags.go:64] FLAG: --registry-qps="5"
Jul 15 10:45:22.432608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426640    2536 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426643    2536 flags.go:64] FLAG: --reserved-memory=""
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426646    2536 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426649    2536 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426652    2536 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426655    2536 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426657    2536 flags.go:64] FLAG: --runonce="false"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426660    2536 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426663    2536 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426665    2536 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426668    2536 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426671    2536 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426674    2536 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426676    2536 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426679    2536 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426682    2536 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426687    2536 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426690    2536 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426692    2536 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426695    2536 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426698    2536 flags.go:64] FLAG: --system-cgroups=""
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426701    2536 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426710    2536 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426718    2536 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426721    2536 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 10:45:22.433217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426725    2536 flags.go:64] FLAG: --tls-min-version=""
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426727    2536 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426730    2536 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426733    2536 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426735    2536 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426738    2536 flags.go:64] FLAG: --v="2"
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426742    2536 flags.go:64] FLAG: --version="false"
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426745    2536 flags.go:64] FLAG: --vmodule=""
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426749    2536 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.426752    2536 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426896    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426900    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426903    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426906    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426908    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426911    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426913    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426916    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426918    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426921    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426923    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426926    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 10:45:22.433884 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426928    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426934    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426937    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426939    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426941    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426944    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426946    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426949    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426951    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426954    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426956    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426959    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426961    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426964    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426966    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426969    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426971    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426974    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426976    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 10:45:22.434429 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426978    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426981    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426984    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426986    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426989    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426991    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426993    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426996    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.426998    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427000    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427003    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427007    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427010    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427014    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427020    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427023    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427026    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427028    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427030    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 10:45:22.434882 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427033    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427035    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427038    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427040    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427043    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427047    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427049    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427051    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427054    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427056    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427059    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427061    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427064    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427066    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427069    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427071    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427073    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427076    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427078    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 10:45:22.435408 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427080    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427084    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427088    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427091    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427094    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427096    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427099    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427101    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427106    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427109    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427111    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427114    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427117    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427119    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427121    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427124    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427126    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427129    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427132    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427134    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 10:45:22.435863 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427137    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427139    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427142    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427144    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427147    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427149    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427151    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427154    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427156    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.427158    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.427647    2536 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.434391    2536 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.434407    2536 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.434421    2536 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.434426    2536 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 10:45:22.436441 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434488    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434492    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434495    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434498    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434502    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434504    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434507    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434509    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434511    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434514    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434516    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434519    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434522    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434524    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434527    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434530    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434532    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434534    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434537    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434539    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 10:45:22.437032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434542    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434544    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434546    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434549    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434551    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434553    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434556    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434558    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434561    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434571    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434574    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434577    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434581    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434584    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434587    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434590    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434593    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434596    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434598    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434601    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 10:45:22.437575 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434603    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434606    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434608    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434611    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434613    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434616    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434618    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434621    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434623    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434627    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434630    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434633    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434636    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434638    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434641    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434643    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434646    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434648    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434650    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 10:45:22.438059 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434653    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434656    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434658    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434666    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434669    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434671    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434674    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434677    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434679    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434682    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434684    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434686    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434689    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434691    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434693    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434696    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434698    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434701    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434703    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434706    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 10:45:22.438538 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434708    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434710    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434713    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434715    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434717    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434720    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434722    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434724    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434727    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434729    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434732    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434734    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434736    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434739    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434742    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434744    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434752    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434755    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434757    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 10:45:22.439033 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434761    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.434766    2536 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434911    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434916    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434918    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434921    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434923    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434926    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434928    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434931    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434933    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434936    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434938    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434940    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434943    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 10:45:22.439663 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434945    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434947    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434950    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434952    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434955    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434957    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434959    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434962    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434964    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434967    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434969    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434972    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434974    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434977    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434979    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434987    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434990    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434992    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434994    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 10:45:22.440032 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.434997    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435000    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435002    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435005    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435007    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435010    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435012    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435014    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435017    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435019    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435022    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435024    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435028    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435030    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435033    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435035    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435039    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435043    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435045    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 10:45:22.440543 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435048    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435051    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435053    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435056    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435059    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435063    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435066    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435069    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435072    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435075    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435083    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435085    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435088    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435091    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435093    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435095    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435098    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435100    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435102    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 10:45:22.441039 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435105    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435107    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435109    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435112    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435114    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435117    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435119    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435121    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435124    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435126    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435128    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435131    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435133    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435135    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435138    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435141    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435143    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435146    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435148    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435150    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 10:45:22.441517 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435153    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435155    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435157    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435160    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435167    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435170    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435172    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435174    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:22.435177    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.435181    2536 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.435318    2536 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.440268    2536 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.441472    2536 server.go:1019] "Starting client certificate rotation"
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.441594    2536 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 10:45:22.442107 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.441647    2536 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 10:45:22.444080 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.444061    2536 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 10:45:22.446454 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.446437    2536 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 10:45:22.453401 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.453385    2536 log.go:25] "Validated CRI v1 runtime API"
Jul 15 10:45:22.456470 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.456455    2536 log.go:25] "Validated CRI v1 image API"
Jul 15 10:45:22.457933 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.457919    2536 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 10:45:22.459844 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.459818    2536 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a8e922d4-8bc2-4fbf-842e-9154f3ca4bf7:/dev/nvme0n1p3 c8b34649-c29d-4510-a16d-b417c959f6d1:/dev/nvme0n1p4]
Jul 15 10:45:22.459915 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.459843    2536 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 10:45:22.464666 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.464555    2536 manager.go:216] Machine: {Timestamp:2026-07-15 10:45:22.46406685 +0000 UTC m=+0.263540366 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103692 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ed094a02c1c3c23b1e4bd7af0c12e SystemUUID:ec2ed094-a02c-1c3c-23b1-e4bd7af0c12e BootID:dd084807-c5ae-4163-a14f-7057ec1dca26 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:54:35:65:7e:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:54:35:65:7e:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:e8:82:99:e3:3f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 10:45:22.464666 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.464651    2536 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 10:45:22.464872 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.464753    2536 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 10:45:22.467390 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.467358    2536 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 10:45:22.467550 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.467394    2536 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-178.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 10:45:22.468474 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.468461    2536 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 10:45:22.468530 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.468478    2536 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 10:45:22.468530 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.468488    2536 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 10:45:22.469434 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.469380    2536 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 10:45:22.470729 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.470710    2536 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 10:45:22.473312 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.473293    2536 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 10:45:22.475031 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.475017    2536 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 10:45:22.475174 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.475164    2536 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 10:45:22.475208 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.475194    2536 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 10:45:22.475208 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.475204    2536 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 10:45:22.475266 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.475214    2536 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 10:45:22.475266 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.475223    2536 kubelet.go:398] "Adding apiserver pod source"
Jul 15 10:45:22.475266 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.475230    2536 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 10:45:22.475621 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.475605    2536 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 10:45:22.475675 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.475625    2536 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 10:45:22.477259 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.477243    2536 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 10:45:22.478970 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.478958    2536 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 10:45:22.479016 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.478988    2536 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 10:45:22.479199 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479189    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 10:45:22.479234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479204    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 10:45:22.479234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479211    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 10:45:22.479234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479216    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 10:45:22.479234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479224    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 10:45:22.479234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479229    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 10:45:22.479693 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479684    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 10:45:22.479725 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479694    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 10:45:22.479725 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479700    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 10:45:22.479725 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479706    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 10:45:22.479725 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479714    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 10:45:22.479725 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.479721    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 10:45:22.480343 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.480333    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 10:45:22.480343 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.480343    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 10:45:22.483587 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.483575    2536 server.go:1268] "Started kubelet"
Jul 15 10:45:22.483674 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.483650    2536 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 10:45:22.483761 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.483698    2536 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 10:45:22.483884 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.483869    2536 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 10:45:22.484433 ip-10-0-134-178 systemd[1]: Started Kubernetes Kubelet.
Jul 15 10:45:22.485119 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.484983    2536 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 10:45:22.485182 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.485168    2536 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-178.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 10:45:22.485517 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.485462    2536 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 10:45:22.485517 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.485464    2536 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 10:45:22.485879 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.485863    2536 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 10:45:22.488218 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.488199    2536 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 10:45:22.490196 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.490181    2536 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 10:45:22.490196 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.490193    2536 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 10:45:22.490422 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.490378    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 15 10:45:22.490979 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.490962    2536 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 10:45:22.491071 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.491009    2536 factory.go:55] Registering systemd factory
Jul 15 10:45:22.491071 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.491023    2536 factory.go:223] Registration of the systemd container factory successfully
Jul 15 10:45:22.491268 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.491253    2536 factory.go:153] Registering CRI-O factory
Jul 15 10:45:22.491375 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.491293    2536 factory.go:223] Registration of the crio container factory successfully
Jul 15 10:45:22.491375 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.491300    2536 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 10:45:22.491375 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.491360    2536 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 10:45:22.491375 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.491372    2536 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 10:45:22.491613 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.491397    2536 factory.go:103] Registering Raw factory
Jul 15 10:45:22.491613 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.491427    2536 manager.go:1196] Started watching for new ooms in manager
Jul 15 10:45:22.491613 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.491481    2536 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 10:45:22.491613 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.491489    2536 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 10:45:22.491788 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.490888    2536 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-178.ec2.internal.18c270b46507740d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-178.ec2.internal,UID:ip-10-0-134-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-178.ec2.internal,},FirstTimestamp:2026-07-15 10:45:22.483549197 +0000 UTC m=+0.283022713,LastTimestamp:2026-07-15 10:45:22.483549197 +0000 UTC m=+0.283022713,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-178.ec2.internal,}"
Jul 15 10:45:22.492320 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.492305    2536 manager.go:349] Starting recovery of all containers
Jul 15 10:45:22.501365 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.501252    2536 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 10:45:22.501365 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.501270    2536 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 10:45:22.504770 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.504581    2536 manager.go:354] Recovery completed
Jul 15 10:45:22.506209 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.506191    2536 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 15 10:45:22.509209 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.509193    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 10:45:22.511591 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.511575    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 10:45:22.511649 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.511605    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 10:45:22.511649 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.511614    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 15 10:45:22.512157 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.512142    2536 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 10:45:22.512157 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.512154    2536 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 10:45:22.512287 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.512173    2536 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 10:45:22.512287 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.512155    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 10:45:22.512973 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.512960    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 10:45:22.513018 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.512984    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 10:45:22.513018 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.512994    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 15 10:45:22.514128 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.514051    2536 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-178.ec2.internal.18c270b466b35b45  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-178.ec2.internal,UID:ip-10-0-134-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-178.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-178.ec2.internal,},FirstTimestamp:2026-07-15 10:45:22.511592261 +0000 UTC m=+0.311065779,LastTimestamp:2026-07-15 10:45:22.511592261 +0000 UTC m=+0.311065779,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-178.ec2.internal,}"
Jul 15 10:45:22.514341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.514330    2536 policy_none.go:49] "None policy: Start"
Jul 15 10:45:22.514375 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.514346    2536 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 10:45:22.514375 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.514356    2536 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 10:45:22.516348 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.516336    2536 policy_none.go:47] "Start"
Jul 15 10:45:22.525055 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.524988    2536 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-178.ec2.internal.18c270b466b39e00  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-178.ec2.internal,UID:ip-10-0-134-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-178.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-178.ec2.internal,},FirstTimestamp:2026-07-15 10:45:22.511609344 +0000 UTC m=+0.311082860,LastTimestamp:2026-07-15 10:45:22.511609344 +0000 UTC m=+0.311082860,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-178.ec2.internal,}"
Jul 15 10:45:22.535615 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.535591    2536 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4tbwf"
Jul 15 10:45:22.535788 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.535734    2536 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-178.ec2.internal.18c270b466b3c11f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-178.ec2.internal,UID:ip-10-0-134-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-178.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-178.ec2.internal,},FirstTimestamp:2026-07-15 10:45:22.511618335 +0000 UTC m=+0.311091851,LastTimestamp:2026-07-15 10:45:22.511618335 +0000 UTC m=+0.311091851,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-178.ec2.internal,}"
Jul 15 10:45:22.544181 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.544163    2536 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4tbwf"
Jul 15 10:45:22.544245 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.544160    2536 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-178.ec2.internal.18c270b466b35b45\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-178.ec2.internal.18c270b466b35b45  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-178.ec2.internal,UID:ip-10-0-134-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-178.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-178.ec2.internal,},FirstTimestamp:2026-07-15 10:45:22.511592261 +0000 UTC m=+0.311065779,LastTimestamp:2026-07-15 10:45:22.512972717 +0000 UTC m=+0.312446233,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-178.ec2.internal,}"
Jul 15 10:45:22.549829 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.549814    2536 manager.go:337] "Starting Device Plugin manager"
Jul 15 10:45:22.549928 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.549847    2536 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 10:45:22.549928 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.549861    2536 server.go:87] "Starting device plugin registration server"
Jul 15 10:45:22.550110 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.550097    2536 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 10:45:22.550164 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.550112    2536 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 10:45:22.550222 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.550187    2536 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 10:45:22.550290 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.550267    2536 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 10:45:22.550348 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.550293    2536 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 10:45:22.550885 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.550855    2536 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 10:45:22.550943 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.550914    2536 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 15 10:45:22.587005 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.586976    2536 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 10:45:22.595082 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.588234    2536 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 10:45:22.595082 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.588267    2536 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 10:45:22.595082 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.588758    2536 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 10:45:22.595082 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.588797    2536 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 10:45:22.595082 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.591413    2536 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 10:45:22.650645 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.650611    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 10:45:22.651454 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.651438    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 10:45:22.651540 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.651470    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 10:45:22.651540 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.651486    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 15 10:45:22.651540 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.651507    2536 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.661433 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.661420    2536 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.661476 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.661441    2536 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-178.ec2.internal\": node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 15 10:45:22.689857 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.689814    2536 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"]
Jul 15 10:45:22.689919 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.689870    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 10:45:22.691098 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.691085    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 10:45:22.691171 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.691110    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 10:45:22.691171 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.691120    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 15 10:45:22.691443 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.691427    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 15 10:45:22.692470 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.692454    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f97328aeac47e4524015111ec9ef9b68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"f97328aeac47e4524015111ec9ef9b68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.692517 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.692478    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f97328aeac47e4524015111ec9ef9b68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"f97328aeac47e4524015111ec9ef9b68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.693446 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.693435    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 10:45:22.693579 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.693564    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.693650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.693600    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 10:45:22.694985 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.694971    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 10:45:22.695072 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.695004    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 10:45:22.695072 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.695019    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 15 10:45:22.695451 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.695437    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 10:45:22.695516 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.695466    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 10:45:22.695516 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.695480    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 15 10:45:22.697868 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.697852    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.697944 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.697877    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 10:45:22.698643 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.698627    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 10:45:22.698721 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.698656    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 10:45:22.698721 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.698666    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeHasSufficientPID"
Jul 15 10:45:22.720835 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.720818    2536 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-178.ec2.internal\" not found" node="ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.725021 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.725005    2536 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-178.ec2.internal\" not found" node="ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.792152 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.792129    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 15 10:45:22.793241 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.793223    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f97328aeac47e4524015111ec9ef9b68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"f97328aeac47e4524015111ec9ef9b68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.793301 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.793250    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f97328aeac47e4524015111ec9ef9b68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"f97328aeac47e4524015111ec9ef9b68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.793301 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.793268    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/da1d1b399f6297ad7ed288ac69e2c3f1-config\") pod \"kube-apiserver-proxy-ip-10-0-134-178.ec2.internal\" (UID: \"da1d1b399f6297ad7ed288ac69e2c3f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.793367 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.793320    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f97328aeac47e4524015111ec9ef9b68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"f97328aeac47e4524015111ec9ef9b68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.793367 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.793332    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f97328aeac47e4524015111ec9ef9b68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal\" (UID: \"f97328aeac47e4524015111ec9ef9b68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.892728 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.892703    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 15 10:45:22.893818 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.893799    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/da1d1b399f6297ad7ed288ac69e2c3f1-config\") pod \"kube-apiserver-proxy-ip-10-0-134-178.ec2.internal\" (UID: \"da1d1b399f6297ad7ed288ac69e2c3f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.893869 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:22.893838    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/da1d1b399f6297ad7ed288ac69e2c3f1-config\") pod \"kube-apiserver-proxy-ip-10-0-134-178.ec2.internal\" (UID: \"da1d1b399f6297ad7ed288ac69e2c3f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:22.993418 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:22.993370    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 15 10:45:23.022616 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.022598    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:23.027406 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.027254    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:23.094124 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.094096    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 15 10:45:23.194528 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.194506    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 15 10:45:23.295033 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.294983    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-178.ec2.internal\" not found"
Jul 15 10:45:23.334521 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.334507    2536 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 10:45:23.390224 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.390203    2536 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 10:45:23.391186 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.391170    2536 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:23.407140 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.407119    2536 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 10:45:23.408597 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.408582    2536 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal"
Jul 15 10:45:23.417994 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.417979    2536 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 10:45:23.441100 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.441083    2536 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 10:45:23.441206 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.441190    2536 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 10:45:23.441257 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.441239    2536 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 10:45:23.441336 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.441256    2536 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 10:45:23.476376 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.476359    2536 apiserver.go:52] "Watching apiserver"
Jul 15 10:45:23.490480 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.490458    2536 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 10:45:23.490788 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.490765    2536 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 10:45:23.491901 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.491877    2536 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 10:45:23.492443 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.492419    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-gx9mp","openshift-multus/multus-additional-cni-plugins-rjftd","openshift-multus/multus-nq8j4","kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal","openshift-multus/network-metrics-daemon-6tndv","openshift-network-diagnostics/network-check-target-n9fpd","openshift-network-operator/iptables-alerter-slr8n","openshift-ovn-kubernetes/ovnkube-node-kbz5v","kube-system/global-pull-secret-syncer-cq9p2","kube-system/konnectivity-agent-rbbsd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x","openshift-cluster-node-tuning-operator/tuned-xlc9x","openshift-dns/node-resolver-s986k"]
Jul 15 10:45:23.495050 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.495026    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:23.495115 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.495089    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:23.497204 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.497181    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:23.499409 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.499389    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.499518 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.499467    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 10:45:23.499637 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.499398    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 10:45:23.499824 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.499810    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qlf7m\""
Jul 15 10:45:23.501523 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.501508    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s986k"
Jul 15 10:45:23.501773 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.501756    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 10:45:23.501897 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.501877    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 10:45:23.502022 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.501977    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 10:45:23.502486 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.502470    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 10:45:23.503692 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.503676    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-slr8n"
Jul 15 10:45:23.504405 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.504386    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 10:45:23.504497 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.504432    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-njdkb\""
Jul 15 10:45:23.505990 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.505975    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:23.506074 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.506041    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:23.508461 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.508444    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.508751 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.508644    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 10:45:23.509482 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.509462    2536 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 10:45:23.510255 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.510235    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7mrhv\""
Jul 15 10:45:23.510526 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.510510    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 10:45:23.510878 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.510858    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.511389 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.511369    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 10:45:23.511479 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.511464    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dph4m\""
Jul 15 10:45:23.511596 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.511582    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 10:45:23.512439 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.512425    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 10:45:23.513149 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.513132    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.515485 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.515467    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gx9mp"
Jul 15 10:45:23.517730 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.517707    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:23.517804 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.517780    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:23.519658 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.519633    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 10:45:23.519793 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.519657    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 10:45:23.519793 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.519734    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 10:45:23.519793 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.519746    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 10:45:23.519793 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.519784    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 10:45:23.519985 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.519865    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 10:45:23.519985 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.519979    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 10:45:23.520082 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.519986    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gf6sq\""
Jul 15 10:45:23.520082 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.519992    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 10:45:23.520082 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.519999    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 10:45:23.520082 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.520043    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 10:45:23.520082 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.520054    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 10:45:23.520269 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.520204    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tq4mh\""
Jul 15 10:45:23.520374 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.520299    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.522196 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.522176    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mr2gx\""
Jul 15 10:45:23.522371 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.522354    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dgdqb\""
Jul 15 10:45:23.522575 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.522558    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 10:45:23.526191 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.526172    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 10:45:23.526655 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.526641    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 10:45:23.526977 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.526957    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dkkn2\""
Jul 15 10:45:23.527055 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.526988    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 10:45:23.548301 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.548221    2536 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 10:40:22 +0000 UTC" deadline="2027-12-08 18:31:33.363613753 +0000 UTC"
Jul 15 10:45:23.548301 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.548264    2536 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12271h46m9.815352458s"
Jul 15 10:45:23.554635 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.554614    2536 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-q4p9d"
Jul 15 10:45:23.568240 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.568224    2536 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-q4p9d"
Jul 15 10:45:23.591721 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.591708    2536 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 10:45:23.596750 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.596734    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-cni-binary-copy\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.596824 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.596764    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-var-lib-openvswitch\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.596824 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.596797    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.596824 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.596816    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-socket-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.596949 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.596842    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wjht\" (UniqueName: \"kubernetes.io/projected/c29a5184-864a-4a82-99a7-308aa7edc863-kube-api-access-8wjht\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.596949 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.596864    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/764e82f9-85c4-44bb-9a80-72c5e55a953b-tmp-dir\") pod \"node-resolver-s986k\" (UID: \"764e82f9-85c4-44bb-9a80-72c5e55a953b\") " pod="openshift-dns/node-resolver-s986k"
Jul 15 10:45:23.596949 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.596901    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dbc90a95-67fa-4fb4-b4f4-20549cbf6194-iptables-alerter-script\") pod \"iptables-alerter-slr8n\" (UID: \"dbc90a95-67fa-4fb4-b4f4-20549cbf6194\") " pod="openshift-network-operator/iptables-alerter-slr8n"
Jul 15 10:45:23.596949 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.596943    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7950898a-8257-4b97-9fed-66ceafd63b56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.597103 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.596968    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-var-lib-kubelet\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.597103 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.596989    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-var-lib-cni-multus\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.597103 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597011    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-run-openvswitch\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.597103 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597032    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-log-socket\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.597103 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597052    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ef2acd01-96a6-43ff-b327-da4636c43ccd-ovnkube-script-lib\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.597103 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597083    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-sys\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.597339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597109    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-system-cni-dir\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.597339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597128    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-cnibin\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.597339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597176    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-slash\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.597339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597210    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-systemd\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.597339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597230    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf-host\") pod \"node-ca-gx9mp\" (UID: \"9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf\") " pod="openshift-image-registry/node-ca-gx9mp"
Jul 15 10:45:23.597339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597243    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-var-lib-cni-bin\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.597339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597294    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.597339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597314    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/764e82f9-85c4-44bb-9a80-72c5e55a953b-hosts-file\") pod \"node-resolver-s986k\" (UID: \"764e82f9-85c4-44bb-9a80-72c5e55a953b\") " pod="openshift-dns/node-resolver-s986k"
Jul 15 10:45:23.597339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597327    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-system-cni-dir\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597345    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7950898a-8257-4b97-9fed-66ceafd63b56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597359    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-etc-kubernetes\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597388    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-tuned\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597414    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b01b337-70ce-4090-b7c9-2756883ff47a-tmp\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597437    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdkr8\" (UniqueName: \"kubernetes.io/projected/764e82f9-85c4-44bb-9a80-72c5e55a953b-kube-api-access-wdkr8\") pod \"node-resolver-s986k\" (UID: \"764e82f9-85c4-44bb-9a80-72c5e55a953b\") " pod="openshift-dns/node-resolver-s986k"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597470    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-cnibin\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597506    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-hostroot\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597530    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khgnd\" (UniqueName: \"kubernetes.io/projected/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-kube-api-access-khgnd\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597547    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-run-ovn\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597572    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-sys-fs\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597598    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-run-systemd\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597626    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-run-ovn-kubernetes\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.597650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597651    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ef2acd01-96a6-43ff-b327-da4636c43ccd-ovn-node-metrics-cert\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597665    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-device-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597681    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf-serviceca\") pod \"node-ca-gx9mp\" (UID: \"9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf\") " pod="openshift-image-registry/node-ca-gx9mp"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597701    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-socket-dir-parent\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597721    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-conf-dir\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597737    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-run-multus-certs\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597749    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bdrs\" (UniqueName: \"kubernetes.io/projected/ef2acd01-96a6-43ff-b327-da4636c43ccd-kube-api-access-4bdrs\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597765    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-var-lib-kubelet\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597784    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-node-log\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597827    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-cni-netd\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597864    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-run\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597877    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dbc90a95-67fa-4fb4-b4f4-20549cbf6194-host-slash\") pod \"iptables-alerter-slr8n\" (UID: \"dbc90a95-67fa-4fb4-b4f4-20549cbf6194\") " pod="openshift-network-operator/iptables-alerter-slr8n"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597890    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npk5s\" (UniqueName: \"kubernetes.io/projected/dbc90a95-67fa-4fb4-b4f4-20549cbf6194-kube-api-access-npk5s\") pod \"iptables-alerter-slr8n\" (UID: \"dbc90a95-67fa-4fb4-b4f4-20549cbf6194\") " pod="openshift-network-operator/iptables-alerter-slr8n"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597909    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-os-release\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597924    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-run-netns\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597938    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-daemon-config\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597949    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-run-netns\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.598091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597961    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ef2acd01-96a6-43ff-b327-da4636c43ccd-ovnkube-config\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.597981    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-modprobe-d\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598013    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng8xf\" (UniqueName: \"kubernetes.io/projected/9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf-kube-api-access-ng8xf\") pod \"node-ca-gx9mp\" (UID: \"9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf\") " pod="openshift-image-registry/node-ca-gx9mp"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598059    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmxj9\" (UniqueName: \"kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9\") pod \"network-check-target-n9fpd\" (UID: \"28145093-25ab-4dd4-9ce2-edf6ea14ca27\") " pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598102    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-cni-dir\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598132    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqj2f\" (UniqueName: \"kubernetes.io/projected/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-kube-api-access-nqj2f\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598164    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-sysctl-d\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598199    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86rxk\" (UniqueName: \"kubernetes.io/projected/8b01b337-70ce-4090-b7c9-2756883ff47a-kube-api-access-86rxk\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598225    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c191b9b2-e426-494e-8dbb-66d9da189ec3-konnectivity-ca\") pod \"konnectivity-agent-rbbsd\" (UID: \"c191b9b2-e426-494e-8dbb-66d9da189ec3\") " pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598255    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-kubelet\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598299    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-etc-openvswitch\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598322    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-sysctl-conf\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598343    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7950898a-8257-4b97-9fed-66ceafd63b56-cni-binary-copy\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598363    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-run-k8s-cni-cncf-io\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598385    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598408    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-systemd-units\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.598789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598431    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ef2acd01-96a6-43ff-b327-da4636c43ccd-env-overrides\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598473    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-host\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598506    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598532    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c191b9b2-e426-494e-8dbb-66d9da189ec3-agent-certs\") pod \"konnectivity-agent-rbbsd\" (UID: \"c191b9b2-e426-494e-8dbb-66d9da189ec3\") " pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598553    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-os-release\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598575    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-kubernetes\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598596    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-kubelet-config\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598616    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598638    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-cni-bin\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598658    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-sysconfig\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598677    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-lib-modules\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598699    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-registration-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598720    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-etc-selinux\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598740    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-dbus\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:23.599444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.598760    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cbx2\" (UniqueName: \"kubernetes.io/projected/7950898a-8257-4b97-9fed-66ceafd63b56-kube-api-access-2cbx2\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.600058 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:23.600036    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1d1b399f6297ad7ed288ac69e2c3f1.slice/crio-79663e96aa0fa64aa9668a648b17c443f358e2dc2a3a065f834bce384d16d18d WatchSource:0}: Error finding container 79663e96aa0fa64aa9668a648b17c443f358e2dc2a3a065f834bce384d16d18d: Status 404 returned error can't find the container with id 79663e96aa0fa64aa9668a648b17c443f358e2dc2a3a065f834bce384d16d18d
Jul 15 10:45:23.600238 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:23.600221    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf97328aeac47e4524015111ec9ef9b68.slice/crio-3f7b999422c899e570a5588ea2daa5d523a0268a13727aa63ad468d1cdf67e43 WatchSource:0}: Error finding container 3f7b999422c899e570a5588ea2daa5d523a0268a13727aa63ad468d1cdf67e43: Status 404 returned error can't find the container with id 3f7b999422c899e570a5588ea2daa5d523a0268a13727aa63ad468d1cdf67e43
Jul 15 10:45:23.604921 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.604903    2536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 10:45:23.699802 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699778    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-hostroot\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.699896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699813    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khgnd\" (UniqueName: \"kubernetes.io/projected/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-kube-api-access-khgnd\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.699896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699843    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-run-ovn\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.699896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699861    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-hostroot\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.699896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699869    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-sys-fs\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.699896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699892    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-run-systemd\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699915    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-run-ovn-kubernetes\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699946    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ef2acd01-96a6-43ff-b327-da4636c43ccd-ovn-node-metrics-cert\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699964    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-run-ovn\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699967    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-device-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700001    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-run-ovn-kubernetes\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700014    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf-serviceca\") pod \"node-ca-gx9mp\" (UID: \"9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf\") " pod="openshift-image-registry/node-ca-gx9mp"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699975    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-run-systemd\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.699972    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-sys-fs\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700004    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-device-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700040    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-socket-dir-parent\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700064    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-conf-dir\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700087    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-run-multus-certs\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.700122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700107    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-conf-dir\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700129    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bdrs\" (UniqueName: \"kubernetes.io/projected/ef2acd01-96a6-43ff-b327-da4636c43ccd-kube-api-access-4bdrs\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700130    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-socket-dir-parent\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700154    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-run-multus-certs\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700164    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-var-lib-kubelet\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700187    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-node-log\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700211    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-cni-netd\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700230    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-run\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700248    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-var-lib-kubelet\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700267    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-node-log\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700306    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dbc90a95-67fa-4fb4-b4f4-20549cbf6194-host-slash\") pod \"iptables-alerter-slr8n\" (UID: \"dbc90a95-67fa-4fb4-b4f4-20549cbf6194\") " pod="openshift-network-operator/iptables-alerter-slr8n"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700312    2536 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700336    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npk5s\" (UniqueName: \"kubernetes.io/projected/dbc90a95-67fa-4fb4-b4f4-20549cbf6194-kube-api-access-npk5s\") pod \"iptables-alerter-slr8n\" (UID: \"dbc90a95-67fa-4fb4-b4f4-20549cbf6194\") " pod="openshift-network-operator/iptables-alerter-slr8n"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700360    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-os-release\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700336    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dbc90a95-67fa-4fb4-b4f4-20549cbf6194-host-slash\") pod \"iptables-alerter-slr8n\" (UID: \"dbc90a95-67fa-4fb4-b4f4-20549cbf6194\") " pod="openshift-network-operator/iptables-alerter-slr8n"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700308    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-cni-netd\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700308    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-run\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700382    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-run-netns\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.700661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700415    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-run-netns\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700423    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-os-release\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700427    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-daemon-config\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700453    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-run-netns\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700456    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf-serviceca\") pod \"node-ca-gx9mp\" (UID: \"9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf\") " pod="openshift-image-registry/node-ca-gx9mp"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700474    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ef2acd01-96a6-43ff-b327-da4636c43ccd-ovnkube-config\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700504    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-modprobe-d\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700513    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-run-netns\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700524    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng8xf\" (UniqueName: \"kubernetes.io/projected/9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf-kube-api-access-ng8xf\") pod \"node-ca-gx9mp\" (UID: \"9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf\") " pod="openshift-image-registry/node-ca-gx9mp"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700546    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxj9\" (UniqueName: \"kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9\") pod \"network-check-target-n9fpd\" (UID: \"28145093-25ab-4dd4-9ce2-edf6ea14ca27\") " pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700623    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-modprobe-d\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700677    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-cni-dir\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700739    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqj2f\" (UniqueName: \"kubernetes.io/projected/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-kube-api-access-nqj2f\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700771    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-sysctl-d\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700795    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86rxk\" (UniqueName: \"kubernetes.io/projected/8b01b337-70ce-4090-b7c9-2756883ff47a-kube-api-access-86rxk\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700794    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-cni-dir\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700881    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c191b9b2-e426-494e-8dbb-66d9da189ec3-konnectivity-ca\") pod \"konnectivity-agent-rbbsd\" (UID: \"c191b9b2-e426-494e-8dbb-66d9da189ec3\") " pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700892    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-multus-daemon-config\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.701494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700919    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-kubelet\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700913    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-sysctl-d\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700955    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-etc-openvswitch\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700975    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-kubelet\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700984    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-sysctl-conf\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701007    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7950898a-8257-4b97-9fed-66ceafd63b56-cni-binary-copy\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701037    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-run-k8s-cni-cncf-io\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701010    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-etc-openvswitch\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.700976    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ef2acd01-96a6-43ff-b327-da4636c43ccd-ovnkube-config\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701082    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701091    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-sysctl-conf\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701105    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-systemd-units\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701110    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-run-k8s-cni-cncf-io\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701126    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ef2acd01-96a6-43ff-b327-da4636c43ccd-env-overrides\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701138    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-systemd-units\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701147    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-host\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701170    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.701182    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:23.702234 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701192    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c191b9b2-e426-494e-8dbb-66d9da189ec3-agent-certs\") pod \"konnectivity-agent-rbbsd\" (UID: \"c191b9b2-e426-494e-8dbb-66d9da189ec3\") " pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701215    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-os-release\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701216    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-host\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.701254    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs podName:4f99bc36-1c09-41cc-9960-e8cf593e3e9c nodeName:}" failed. No retries permitted until 2026-07-15 10:45:24.201219205 +0000 UTC m=+2.000692707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs") pod "network-metrics-daemon-6tndv" (UID: "4f99bc36-1c09-41cc-9960-e8cf593e3e9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701304    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-os-release\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701354    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-kubernetes\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701379    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-kubelet-config\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701401    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701403    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c191b9b2-e426-494e-8dbb-66d9da189ec3-konnectivity-ca\") pod \"konnectivity-agent-rbbsd\" (UID: \"c191b9b2-e426-494e-8dbb-66d9da189ec3\") " pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701423    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-cni-bin\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701440    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-kubernetes\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701454    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-kubelet-config\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701463    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-sysconfig\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701487    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-lib-modules\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701492    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-cni-bin\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701511    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-registration-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.702993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701527    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-sysconfig\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701532    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-etc-selinux\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701488    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ef2acd01-96a6-43ff-b327-da4636c43ccd-env-overrides\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701554    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-dbus\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701565    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701577    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cbx2\" (UniqueName: \"kubernetes.io/projected/7950898a-8257-4b97-9fed-66ceafd63b56-kube-api-access-2cbx2\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701587    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-registration-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701589    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-lib-modules\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701605    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7950898a-8257-4b97-9fed-66ceafd63b56-cni-binary-copy\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701600    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-cni-binary-copy\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701632    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-dbus\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701636    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-etc-selinux\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701656    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-var-lib-openvswitch\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701678    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701728    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-socket-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701744    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-var-lib-openvswitch\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.701755    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:23.703677 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.701751    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wjht\" (UniqueName: \"kubernetes.io/projected/c29a5184-864a-4a82-99a7-308aa7edc863-kube-api-access-8wjht\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.702330    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-socket-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.702386    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c29a5184-864a-4a82-99a7-308aa7edc863-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.702430    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret podName:f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:24.202406136 +0000 UTC m=+2.001879651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret") pod "global-pull-secret-syncer-cq9p2" (UID: "f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5") : object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.702508    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-cni-binary-copy\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703358    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/764e82f9-85c4-44bb-9a80-72c5e55a953b-tmp-dir\") pod \"node-resolver-s986k\" (UID: \"764e82f9-85c4-44bb-9a80-72c5e55a953b\") " pod="openshift-dns/node-resolver-s986k"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703643    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/764e82f9-85c4-44bb-9a80-72c5e55a953b-tmp-dir\") pod \"node-resolver-s986k\" (UID: \"764e82f9-85c4-44bb-9a80-72c5e55a953b\") " pod="openshift-dns/node-resolver-s986k"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703683    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dbc90a95-67fa-4fb4-b4f4-20549cbf6194-iptables-alerter-script\") pod \"iptables-alerter-slr8n\" (UID: \"dbc90a95-67fa-4fb4-b4f4-20549cbf6194\") " pod="openshift-network-operator/iptables-alerter-slr8n"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703712    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7950898a-8257-4b97-9fed-66ceafd63b56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703741    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-var-lib-kubelet\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703767    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-var-lib-cni-multus\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703789    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-run-openvswitch\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703814    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-log-socket\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703841    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ef2acd01-96a6-43ff-b327-da4636c43ccd-ovnkube-script-lib\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703866    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-sys\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703886    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ef2acd01-96a6-43ff-b327-da4636c43ccd-ovn-node-metrics-cert\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703978    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c191b9b2-e426-494e-8dbb-66d9da189ec3-agent-certs\") pod \"konnectivity-agent-rbbsd\" (UID: \"c191b9b2-e426-494e-8dbb-66d9da189ec3\") " pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:23.704445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703893    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-system-cni-dir\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704045    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-cnibin\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704068    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-slash\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704094    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-systemd\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704120    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf-host\") pod \"node-ca-gx9mp\" (UID: \"9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf\") " pod="openshift-image-registry/node-ca-gx9mp"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704145    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-var-lib-cni-bin\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704172    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704209    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/764e82f9-85c4-44bb-9a80-72c5e55a953b-hosts-file\") pod \"node-resolver-s986k\" (UID: \"764e82f9-85c4-44bb-9a80-72c5e55a953b\") " pod="openshift-dns/node-resolver-s986k"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704235    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-system-cni-dir\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704285    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7950898a-8257-4b97-9fed-66ceafd63b56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704314    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-etc-kubernetes\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704340    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-tuned\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704362    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b01b337-70ce-4090-b7c9-2756883ff47a-tmp\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704369    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-var-lib-kubelet\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704391    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdkr8\" (UniqueName: \"kubernetes.io/projected/764e82f9-85c4-44bb-9a80-72c5e55a953b-kube-api-access-wdkr8\") pod \"node-resolver-s986k\" (UID: \"764e82f9-85c4-44bb-9a80-72c5e55a953b\") " pod="openshift-dns/node-resolver-s986k"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704412    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-var-lib-cni-multus\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704419    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-cnibin\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.705147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.703985    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-system-cni-dir\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704527    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704530    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf-host\") pod \"node-ca-gx9mp\" (UID: \"9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf\") " pod="openshift-image-registry/node-ca-gx9mp"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704526    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/764e82f9-85c4-44bb-9a80-72c5e55a953b-hosts-file\") pod \"node-resolver-s986k\" (UID: \"764e82f9-85c4-44bb-9a80-72c5e55a953b\") " pod="openshift-dns/node-resolver-s986k"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704564    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-run-openvswitch\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704598    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-systemd\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704601    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-log-socket\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704646    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef2acd01-96a6-43ff-b327-da4636c43ccd-host-slash\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704690    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-cnibin\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704731    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-etc-kubernetes\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704895    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-cnibin\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.704962    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b01b337-70ce-4090-b7c9-2756883ff47a-sys\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.705051    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7950898a-8257-4b97-9fed-66ceafd63b56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.705061    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dbc90a95-67fa-4fb4-b4f4-20549cbf6194-iptables-alerter-script\") pod \"iptables-alerter-slr8n\" (UID: \"dbc90a95-67fa-4fb4-b4f4-20549cbf6194\") " pod="openshift-network-operator/iptables-alerter-slr8n"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.705066    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-host-var-lib-cni-bin\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.705116    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7950898a-8257-4b97-9fed-66ceafd63b56-system-cni-dir\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.705696    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7950898a-8257-4b97-9fed-66ceafd63b56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.706339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.706025    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ef2acd01-96a6-43ff-b327-da4636c43ccd-ovnkube-script-lib\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.706849 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.706835    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b01b337-70ce-4090-b7c9-2756883ff47a-tmp\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.706987 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.706970    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8b01b337-70ce-4090-b7c9-2756883ff47a-etc-tuned\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.733145 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.733120    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wjht\" (UniqueName: \"kubernetes.io/projected/c29a5184-864a-4a82-99a7-308aa7edc863-kube-api-access-8wjht\") pod \"aws-ebs-csi-driver-node-h286x\" (UID: \"c29a5184-864a-4a82-99a7-308aa7edc863\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.733376 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.733362    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqj2f\" (UniqueName: \"kubernetes.io/projected/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-kube-api-access-nqj2f\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:23.737330 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.737318    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 10:45:23.737379 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.737332    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 10:45:23.737379 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.737340    2536 projected.go:196] Error preparing data for projected volume kube-api-access-jmxj9 for pod openshift-network-diagnostics/network-check-target-n9fpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:23.737379 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:23.737377    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9 podName:28145093-25ab-4dd4-9ce2-edf6ea14ca27 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:24.237365562 +0000 UTC m=+2.036839064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jmxj9" (UniqueName: "kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9") pod "network-check-target-n9fpd" (UID: "28145093-25ab-4dd4-9ce2-edf6ea14ca27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:23.739069 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.739055    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cbx2\" (UniqueName: \"kubernetes.io/projected/7950898a-8257-4b97-9fed-66ceafd63b56-kube-api-access-2cbx2\") pod \"multus-additional-cni-plugins-rjftd\" (UID: \"7950898a-8257-4b97-9fed-66ceafd63b56\") " pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.741227 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.741210    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bdrs\" (UniqueName: \"kubernetes.io/projected/ef2acd01-96a6-43ff-b327-da4636c43ccd-kube-api-access-4bdrs\") pod \"ovnkube-node-kbz5v\" (UID: \"ef2acd01-96a6-43ff-b327-da4636c43ccd\") " pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.741588 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.741571    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdkr8\" (UniqueName: \"kubernetes.io/projected/764e82f9-85c4-44bb-9a80-72c5e55a953b-kube-api-access-wdkr8\") pod \"node-resolver-s986k\" (UID: \"764e82f9-85c4-44bb-9a80-72c5e55a953b\") " pod="openshift-dns/node-resolver-s986k"
Jul 15 10:45:23.743254 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.743234    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khgnd\" (UniqueName: \"kubernetes.io/projected/7e64f418-38d1-4fec-ad3e-fe0fd74a9169-kube-api-access-khgnd\") pod \"multus-nq8j4\" (UID: \"7e64f418-38d1-4fec-ad3e-fe0fd74a9169\") " pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.744126 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.744104    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86rxk\" (UniqueName: \"kubernetes.io/projected/8b01b337-70ce-4090-b7c9-2756883ff47a-kube-api-access-86rxk\") pod \"tuned-xlc9x\" (UID: \"8b01b337-70ce-4090-b7c9-2756883ff47a\") " pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.744974 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.744952    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng8xf\" (UniqueName: \"kubernetes.io/projected/9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf-kube-api-access-ng8xf\") pod \"node-ca-gx9mp\" (UID: \"9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf\") " pod="openshift-image-registry/node-ca-gx9mp"
Jul 15 10:45:23.745863 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.745847    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npk5s\" (UniqueName: \"kubernetes.io/projected/dbc90a95-67fa-4fb4-b4f4-20549cbf6194-kube-api-access-npk5s\") pod \"iptables-alerter-slr8n\" (UID: \"dbc90a95-67fa-4fb4-b4f4-20549cbf6194\") " pod="openshift-network-operator/iptables-alerter-slr8n"
Jul 15 10:45:23.823785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.823744    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:23.831673 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.831655    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nq8j4"
Jul 15 10:45:23.833280 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:23.833246    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc191b9b2_e426_494e_8dbb_66d9da189ec3.slice/crio-f02a141d8f83596c62977dc31b3f357deaa015c5cb15594670328e126169dc79 WatchSource:0}: Error finding container f02a141d8f83596c62977dc31b3f357deaa015c5cb15594670328e126169dc79: Status 404 returned error can't find the container with id f02a141d8f83596c62977dc31b3f357deaa015c5cb15594670328e126169dc79
Jul 15 10:45:23.839001 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:23.838984    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e64f418_38d1_4fec_ad3e_fe0fd74a9169.slice/crio-acbfee193a1a5eb7ba12f23a7c46a6e20bcc3d42e8e4d5c190ee38c93bf908f8 WatchSource:0}: Error finding container acbfee193a1a5eb7ba12f23a7c46a6e20bcc3d42e8e4d5c190ee38c93bf908f8: Status 404 returned error can't find the container with id acbfee193a1a5eb7ba12f23a7c46a6e20bcc3d42e8e4d5c190ee38c93bf908f8
Jul 15 10:45:23.854704 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.854687    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s986k"
Jul 15 10:45:23.864246 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:23.864229    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod764e82f9_85c4_44bb_9a80_72c5e55a953b.slice/crio-053a08eb3b0c5cfc14e82b11be656174da8430ba92e7c7532fc4b6aa9b62194a WatchSource:0}: Error finding container 053a08eb3b0c5cfc14e82b11be656174da8430ba92e7c7532fc4b6aa9b62194a: Status 404 returned error can't find the container with id 053a08eb3b0c5cfc14e82b11be656174da8430ba92e7c7532fc4b6aa9b62194a
Jul 15 10:45:23.866654 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.866639    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-slr8n"
Jul 15 10:45:23.873231 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:23.873215    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc90a95_67fa_4fb4_b4f4_20549cbf6194.slice/crio-611e1cb3f30890687c3ff61068d64e6f73f839d7005cadbbcb132726eb913b1c WatchSource:0}: Error finding container 611e1cb3f30890687c3ff61068d64e6f73f839d7005cadbbcb132726eb913b1c: Status 404 returned error can't find the container with id 611e1cb3f30890687c3ff61068d64e6f73f839d7005cadbbcb132726eb913b1c
Jul 15 10:45:23.881644 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.881628    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:23.888785 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:23.888769    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef2acd01_96a6_43ff_b327_da4636c43ccd.slice/crio-e775f2136506d212b1c36294b4ed8d191f67d45643db930183299442f067d787 WatchSource:0}: Error finding container e775f2136506d212b1c36294b4ed8d191f67d45643db930183299442f067d787: Status 404 returned error can't find the container with id e775f2136506d212b1c36294b4ed8d191f67d45643db930183299442f067d787
Jul 15 10:45:23.902862 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.902841    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xlc9x"
Jul 15 10:45:23.907346 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.907331    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rjftd"
Jul 15 10:45:23.909449 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:23.909433    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b01b337_70ce_4090_b7c9_2756883ff47a.slice/crio-f92bfbc10813d974f48bc468452cd6fe36888615a631c35c660cc968e378ad4d WatchSource:0}: Error finding container f92bfbc10813d974f48bc468452cd6fe36888615a631c35c660cc968e378ad4d: Status 404 returned error can't find the container with id f92bfbc10813d974f48bc468452cd6fe36888615a631c35c660cc968e378ad4d
Jul 15 10:45:23.913036 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.912915    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gx9mp"
Jul 15 10:45:23.914762 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:23.914742    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7950898a_8257_4b97_9fed_66ceafd63b56.slice/crio-cfff8c9874e3f3d142422a7b15a762b2a35222c9d8aa89b98ba3e4ba9a8b843d WatchSource:0}: Error finding container cfff8c9874e3f3d142422a7b15a762b2a35222c9d8aa89b98ba3e4ba9a8b843d: Status 404 returned error can't find the container with id cfff8c9874e3f3d142422a7b15a762b2a35222c9d8aa89b98ba3e4ba9a8b843d
Jul 15 10:45:23.917020 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:23.917003    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x"
Jul 15 10:45:23.921545 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:23.921528    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ad1c6a7_84e8_423b_a1bf_bbb2200a0faf.slice/crio-1a13f7427bf0a21e225e0c9f384f8e0b7fc7c3bf812ecb0010ad2b67ddc24ebb WatchSource:0}: Error finding container 1a13f7427bf0a21e225e0c9f384f8e0b7fc7c3bf812ecb0010ad2b67ddc24ebb: Status 404 returned error can't find the container with id 1a13f7427bf0a21e225e0c9f384f8e0b7fc7c3bf812ecb0010ad2b67ddc24ebb
Jul 15 10:45:23.926400 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:23.926382    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29a5184_864a_4a82_99a7_308aa7edc863.slice/crio-f112f3da828422c75e01b78efa6a76191d651fc693d363863f28f25feedeb19f WatchSource:0}: Error finding container f112f3da828422c75e01b78efa6a76191d651fc693d363863f28f25feedeb19f: Status 404 returned error can't find the container with id f112f3da828422c75e01b78efa6a76191d651fc693d363863f28f25feedeb19f
Jul 15 10:45:24.215585 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.215510    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:24.215585 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.215560    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:24.215765 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:24.215685    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:24.215765 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:24.215737    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret podName:f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:25.215720825 +0000 UTC m=+3.015194330 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret") pod "global-pull-secret-syncer-cq9p2" (UID: "f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5") : object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:24.216161 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:24.216087    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:24.216161 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:24.216132    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs podName:4f99bc36-1c09-41cc-9960-e8cf593e3e9c nodeName:}" failed. No retries permitted until 2026-07-15 10:45:25.216117798 +0000 UTC m=+3.015591305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs") pod "network-metrics-daemon-6tndv" (UID: "4f99bc36-1c09-41cc-9960-e8cf593e3e9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:24.316293 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.316245    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxj9\" (UniqueName: \"kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9\") pod \"network-check-target-n9fpd\" (UID: \"28145093-25ab-4dd4-9ce2-edf6ea14ca27\") " pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:24.316453 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:24.316432    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 10:45:24.316453 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:24.316449    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 10:45:24.316553 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:24.316459    2536 projected.go:196] Error preparing data for projected volume kube-api-access-jmxj9 for pod openshift-network-diagnostics/network-check-target-n9fpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:24.316553 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:24.316508    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9 podName:28145093-25ab-4dd4-9ce2-edf6ea14ca27 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:25.31649244 +0000 UTC m=+3.115965944 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jmxj9" (UniqueName: "kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9") pod "network-check-target-n9fpd" (UID: "28145093-25ab-4dd4-9ce2-edf6ea14ca27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:24.430648 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.430621    2536 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 10:45:24.571565 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.571420    2536 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 10:40:23 +0000 UTC" deadline="2028-05-01 08:49:28.749575846 +0000 UTC"
Jul 15 10:45:24.571565 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.571470    2536 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15742h4m4.17810919s"
Jul 15 10:45:24.598870 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.598780    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal" event={"ID":"f97328aeac47e4524015111ec9ef9b68","Type":"ContainerStarted","Data":"3f7b999422c899e570a5588ea2daa5d523a0268a13727aa63ad468d1cdf67e43"}
Jul 15 10:45:24.604230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.604136    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x" event={"ID":"c29a5184-864a-4a82-99a7-308aa7edc863","Type":"ContainerStarted","Data":"f112f3da828422c75e01b78efa6a76191d651fc693d363863f28f25feedeb19f"}
Jul 15 10:45:24.607219 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.607150    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gx9mp" event={"ID":"9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf","Type":"ContainerStarted","Data":"1a13f7427bf0a21e225e0c9f384f8e0b7fc7c3bf812ecb0010ad2b67ddc24ebb"}
Jul 15 10:45:24.609045 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.608964    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xlc9x" event={"ID":"8b01b337-70ce-4090-b7c9-2756883ff47a","Type":"ContainerStarted","Data":"f92bfbc10813d974f48bc468452cd6fe36888615a631c35c660cc968e378ad4d"}
Jul 15 10:45:24.621093 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.620989    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v" event={"ID":"ef2acd01-96a6-43ff-b327-da4636c43ccd","Type":"ContainerStarted","Data":"e775f2136506d212b1c36294b4ed8d191f67d45643db930183299442f067d787"}
Jul 15 10:45:24.633600 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.633576    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nq8j4" event={"ID":"7e64f418-38d1-4fec-ad3e-fe0fd74a9169","Type":"ContainerStarted","Data":"acbfee193a1a5eb7ba12f23a7c46a6e20bcc3d42e8e4d5c190ee38c93bf908f8"}
Jul 15 10:45:24.641573 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.641549    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rbbsd" event={"ID":"c191b9b2-e426-494e-8dbb-66d9da189ec3","Type":"ContainerStarted","Data":"f02a141d8f83596c62977dc31b3f357deaa015c5cb15594670328e126169dc79"}
Jul 15 10:45:24.649850 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.649826    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjftd" event={"ID":"7950898a-8257-4b97-9fed-66ceafd63b56","Type":"ContainerStarted","Data":"cfff8c9874e3f3d142422a7b15a762b2a35222c9d8aa89b98ba3e4ba9a8b843d"}
Jul 15 10:45:24.655568 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.655491    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-slr8n" event={"ID":"dbc90a95-67fa-4fb4-b4f4-20549cbf6194","Type":"ContainerStarted","Data":"611e1cb3f30890687c3ff61068d64e6f73f839d7005cadbbcb132726eb913b1c"}
Jul 15 10:45:24.660059 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.659968    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s986k" event={"ID":"764e82f9-85c4-44bb-9a80-72c5e55a953b","Type":"ContainerStarted","Data":"053a08eb3b0c5cfc14e82b11be656174da8430ba92e7c7532fc4b6aa9b62194a"}
Jul 15 10:45:24.667313 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:24.667292    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal" event={"ID":"da1d1b399f6297ad7ed288ac69e2c3f1","Type":"ContainerStarted","Data":"79663e96aa0fa64aa9668a648b17c443f358e2dc2a3a065f834bce384d16d18d"}
Jul 15 10:45:25.223292 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:25.222614    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:25.223292 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:25.222660    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:25.223292 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:25.222769    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:25.223292 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:25.222824    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret podName:f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:27.222806102 +0000 UTC m=+5.022279604 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret") pod "global-pull-secret-syncer-cq9p2" (UID: "f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5") : object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:25.223292 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:25.223187    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:25.223292 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:25.223242    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs podName:4f99bc36-1c09-41cc-9960-e8cf593e3e9c nodeName:}" failed. No retries permitted until 2026-07-15 10:45:27.2232267 +0000 UTC m=+5.022700202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs") pod "network-metrics-daemon-6tndv" (UID: "4f99bc36-1c09-41cc-9960-e8cf593e3e9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:25.323853 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:25.323741    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxj9\" (UniqueName: \"kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9\") pod \"network-check-target-n9fpd\" (UID: \"28145093-25ab-4dd4-9ce2-edf6ea14ca27\") " pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:25.324028 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:25.323922    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 10:45:25.324028 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:25.323948    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 10:45:25.324028 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:25.323960    2536 projected.go:196] Error preparing data for projected volume kube-api-access-jmxj9 for pod openshift-network-diagnostics/network-check-target-n9fpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:25.324028 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:25.324016    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9 podName:28145093-25ab-4dd4-9ce2-edf6ea14ca27 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:27.323999114 +0000 UTC m=+5.123472627 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jmxj9" (UniqueName: "kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9") pod "network-check-target-n9fpd" (UID: "28145093-25ab-4dd4-9ce2-edf6ea14ca27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:25.572608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:25.572312    2536 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 10:40:23 +0000 UTC" deadline="2028-01-23 15:48:11.565043986 +0000 UTC"
Jul 15 10:45:25.572608 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:25.572348    2536 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13373h2m45.992700735s"
Jul 15 10:45:25.589020 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:25.588981    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:25.589155 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:25.589111    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:25.589742 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:25.589444    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:25.589742 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:25.589528    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:25.589742 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:25.589592    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:25.589742 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:25.589667    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:25.951466 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:25.951174    2536 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 10:45:26.157458 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:26.157429    2536 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 10:45:27.241993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:27.241829    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:27.241993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:27.241871    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:27.241993 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:27.241967    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:27.242656 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:27.242025    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs podName:4f99bc36-1c09-41cc-9960-e8cf593e3e9c nodeName:}" failed. No retries permitted until 2026-07-15 10:45:31.242006721 +0000 UTC m=+9.041480228 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs") pod "network-metrics-daemon-6tndv" (UID: "4f99bc36-1c09-41cc-9960-e8cf593e3e9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:27.242656 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:27.241967    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:27.242656 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:27.242093    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret podName:f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:31.242081804 +0000 UTC m=+9.041555308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret") pod "global-pull-secret-syncer-cq9p2" (UID: "f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5") : object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:27.343332 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:27.342589    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxj9\" (UniqueName: \"kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9\") pod \"network-check-target-n9fpd\" (UID: \"28145093-25ab-4dd4-9ce2-edf6ea14ca27\") " pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:27.343332 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:27.342782    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 10:45:27.343332 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:27.342801    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 10:45:27.343332 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:27.342814    2536 projected.go:196] Error preparing data for projected volume kube-api-access-jmxj9 for pod openshift-network-diagnostics/network-check-target-n9fpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:27.343332 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:27.342870    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9 podName:28145093-25ab-4dd4-9ce2-edf6ea14ca27 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:31.342852133 +0000 UTC m=+9.142325639 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jmxj9" (UniqueName: "kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9") pod "network-check-target-n9fpd" (UID: "28145093-25ab-4dd4-9ce2-edf6ea14ca27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:27.589911 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:27.589839    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:27.590060 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:27.589964    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:27.590297 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:27.590263    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:27.590386 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:27.590361    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:27.590448 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:27.590408    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:27.590505 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:27.590480    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:29.589268 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:29.589233    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:29.589785 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:29.589376    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:29.589785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:29.589459    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:29.589785 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:29.589540    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:29.589785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:29.589588    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:29.589785 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:29.589652    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:31.276760 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:31.276703    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:31.276760 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:31.276755    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:31.277360 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:31.276924    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:31.277360 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:31.276987    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret podName:f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:39.276967336 +0000 UTC m=+17.076440842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret") pod "global-pull-secret-syncer-cq9p2" (UID: "f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5") : object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:31.277510 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:31.277417    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:31.277510 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:31.277465    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs podName:4f99bc36-1c09-41cc-9960-e8cf593e3e9c nodeName:}" failed. No retries permitted until 2026-07-15 10:45:39.277450401 +0000 UTC m=+17.076923921 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs") pod "network-metrics-daemon-6tndv" (UID: "4f99bc36-1c09-41cc-9960-e8cf593e3e9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:31.377457 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:31.377415    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxj9\" (UniqueName: \"kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9\") pod \"network-check-target-n9fpd\" (UID: \"28145093-25ab-4dd4-9ce2-edf6ea14ca27\") " pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:31.377861 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:31.377652    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 10:45:31.377861 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:31.377696    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 10:45:31.377861 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:31.377710    2536 projected.go:196] Error preparing data for projected volume kube-api-access-jmxj9 for pod openshift-network-diagnostics/network-check-target-n9fpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:31.377861 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:31.377828    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9 podName:28145093-25ab-4dd4-9ce2-edf6ea14ca27 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:39.377786126 +0000 UTC m=+17.177259643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jmxj9" (UniqueName: "kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9") pod "network-check-target-n9fpd" (UID: "28145093-25ab-4dd4-9ce2-edf6ea14ca27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:31.589269 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:31.589190    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:31.589444 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:31.589338    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:31.589444 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:31.589423    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:31.589593 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:31.589539    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:31.589738 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:31.589633    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:31.589792 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:31.589739    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:33.589569 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:33.589489    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:33.589569 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:33.589527    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:33.589569 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:33.589527    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:33.590251 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:33.589626    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:33.590251 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:33.589707    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:33.590251 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:33.589795    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:35.589655 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:35.589482    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:35.590035 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:35.589482    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:35.590035 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:35.589731    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:35.590035 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:35.589804    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:35.590035 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:35.589482    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:35.590035 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:35.589917    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:37.589313 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:37.589269    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:37.589730 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:37.589269    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:37.589730 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:37.589398    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:37.589730 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:37.589486    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:37.589730 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:37.589269    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:37.589730 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:37.589584    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:39.340908 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:39.340872    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:39.340908 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:39.340907    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:39.341432 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:39.341045    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:39.341432 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:39.341048    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:39.341432 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:39.341115    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs podName:4f99bc36-1c09-41cc-9960-e8cf593e3e9c nodeName:}" failed. No retries permitted until 2026-07-15 10:45:55.341099677 +0000 UTC m=+33.140573179 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs") pod "network-metrics-daemon-6tndv" (UID: "4f99bc36-1c09-41cc-9960-e8cf593e3e9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:39.341432 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:39.341129    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret podName:f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:55.341123033 +0000 UTC m=+33.140596536 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret") pod "global-pull-secret-syncer-cq9p2" (UID: "f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5") : object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:39.442180 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:39.442148    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxj9\" (UniqueName: \"kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9\") pod \"network-check-target-n9fpd\" (UID: \"28145093-25ab-4dd4-9ce2-edf6ea14ca27\") " pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:39.442387 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:39.442349    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 10:45:39.442387 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:39.442372    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 10:45:39.442474 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:39.442385    2536 projected.go:196] Error preparing data for projected volume kube-api-access-jmxj9 for pod openshift-network-diagnostics/network-check-target-n9fpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:39.442519 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:39.442475    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9 podName:28145093-25ab-4dd4-9ce2-edf6ea14ca27 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:55.442454352 +0000 UTC m=+33.241927860 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jmxj9" (UniqueName: "kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9") pod "network-check-target-n9fpd" (UID: "28145093-25ab-4dd4-9ce2-edf6ea14ca27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:39.589085 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:39.589055    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:39.589317 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:39.589165    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:39.589317 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:39.589170    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:39.589317 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:39.589181    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:39.589317 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:39.589291    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:39.589494 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:39.589378    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:41.589786 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:41.589741    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:41.590184 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:41.589877    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:41.590184 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:41.589889    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:41.590184 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:41.589963    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:41.590184 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:41.589988    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:41.590184 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:41.590077    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:43.589458 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:43.589428    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:43.589458 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:43.589460    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:43.589907 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:43.589534    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:43.589907 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:43.589429    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:43.589907 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:43.589664    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:43.589907 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:43.589765    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:44.719565 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.719399    2536 generic.go:356] "Generic (PLEG): container finished" podID="7950898a-8257-4b97-9fed-66ceafd63b56" containerID="8c8e29181d1e1eadffef67dff7e0f789b8031d71b11a707b010e8e14699bb8e8" exitCode=0
Jul 15 10:45:44.720186 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.719477    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjftd" event={"ID":"7950898a-8257-4b97-9fed-66ceafd63b56","Type":"ContainerDied","Data":"8c8e29181d1e1eadffef67dff7e0f789b8031d71b11a707b010e8e14699bb8e8"}
Jul 15 10:45:44.721053 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.720863    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s986k" event={"ID":"764e82f9-85c4-44bb-9a80-72c5e55a953b","Type":"ContainerStarted","Data":"74c45427ba53ec795f3fd621bd97b9f862596693588a79f33239877b14078165"}
Jul 15 10:45:44.722366 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.722342    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal" event={"ID":"da1d1b399f6297ad7ed288ac69e2c3f1","Type":"ContainerStarted","Data":"1cdf02c50624d3b6f510a9c41e2e7a804b8844f1089a86bb5bbd5520eeb47b10"}
Jul 15 10:45:44.723628 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.723609    2536 generic.go:356] "Generic (PLEG): container finished" podID="f97328aeac47e4524015111ec9ef9b68" containerID="457cbd466b78a70f1d8202745ab8649d74081ef9b35fcecd054cbaca373f88d1" exitCode=0
Jul 15 10:45:44.723709 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.723661    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal" event={"ID":"f97328aeac47e4524015111ec9ef9b68","Type":"ContainerDied","Data":"457cbd466b78a70f1d8202745ab8649d74081ef9b35fcecd054cbaca373f88d1"}
Jul 15 10:45:44.724921 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.724904    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x" event={"ID":"c29a5184-864a-4a82-99a7-308aa7edc863","Type":"ContainerStarted","Data":"94ac1ec81ad3cb9e6588b161a1d5eab349ad34f4a483a0e26116a401277dd092"}
Jul 15 10:45:44.726208 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.726189    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gx9mp" event={"ID":"9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf","Type":"ContainerStarted","Data":"58ca6210f2619bb35628dd626cef6366537de321b9240ca2f4a0f0e9ab75d59d"}
Jul 15 10:45:44.727416 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.727389    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xlc9x" event={"ID":"8b01b337-70ce-4090-b7c9-2756883ff47a","Type":"ContainerStarted","Data":"0f7ae6dea5621b3735af3abb1e93a912ef8c9e4c973a7e50f52f4dceeafd5395"}
Jul 15 10:45:44.729463 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.729442    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v" event={"ID":"ef2acd01-96a6-43ff-b327-da4636c43ccd","Type":"ContainerStarted","Data":"390cd2473094963d9396ee2bcb55e1b50631d72027e5c0fb4b998ce031ba63f4"}
Jul 15 10:45:44.729557 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.729512    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v" event={"ID":"ef2acd01-96a6-43ff-b327-da4636c43ccd","Type":"ContainerStarted","Data":"a90cc8b625787865aa91b47ee1f5bcf580cf041b165c583f7b54a173d8d2586c"}
Jul 15 10:45:44.729624 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.729565    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v" event={"ID":"ef2acd01-96a6-43ff-b327-da4636c43ccd","Type":"ContainerStarted","Data":"4642970d635c409a99534a7e063d3ee1e808645053bd9a0bc50963259bfa1164"}
Jul 15 10:45:44.729681 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.729622    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v" event={"ID":"ef2acd01-96a6-43ff-b327-da4636c43ccd","Type":"ContainerStarted","Data":"cdde3f9b7f7fc7947ad97f72b1e90033573f36353e732a107d191a225670bc8f"}
Jul 15 10:45:44.730647 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.730630    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nq8j4" event={"ID":"7e64f418-38d1-4fec-ad3e-fe0fd74a9169","Type":"ContainerStarted","Data":"7a42bda514e47cd187374cd293f3304b840f3cf8cf6a4f490f1b23537e34c7a1"}
Jul 15 10:45:44.731785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.731767    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rbbsd" event={"ID":"c191b9b2-e426-494e-8dbb-66d9da189ec3","Type":"ContainerStarted","Data":"fabb03d1492ab16648ec0361df8cd75f5430a0e98bcb2a47227df0a1dff573f2"}
Jul 15 10:45:44.770218 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.770176    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xlc9x" podStartSLOduration=2.593838468 podStartE2EDuration="22.770165051s" totalImagesPullingTime="20.176326583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:45:44.769963298 +0000 UTC m=+22.569436823" watchObservedRunningTime="2026-07-15 10:45:44.770165051 +0000 UTC m=+22.569638575"
Jul 15 10:45:44.810371 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.810318    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gx9mp" podStartSLOduration=3.087305138 podStartE2EDuration="22.810266801s" totalImagesPullingTime="19.722961663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:45:44.785751228 +0000 UTC m=+22.585224751" watchObservedRunningTime="2026-07-15 10:45:44.810266801 +0000 UTC m=+22.609740328"
Jul 15 10:45:44.810594 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.810572    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-178.ec2.internal" podStartSLOduration=21.810564354 podStartE2EDuration="21.810564354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:45:44.808962456 +0000 UTC m=+22.608435987" watchObservedRunningTime="2026-07-15 10:45:44.810564354 +0000 UTC m=+22.610037879"
Jul 15 10:45:44.838981 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.838945    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s986k" podStartSLOduration=2.662660877 podStartE2EDuration="22.838932529s" totalImagesPullingTime="20.176271652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:45:44.838721278 +0000 UTC m=+22.638194802" watchObservedRunningTime="2026-07-15 10:45:44.838932529 +0000 UTC m=+22.638406055"
Jul 15 10:45:44.858412 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.858382    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rbbsd" podStartSLOduration=6.822789329 podStartE2EDuration="22.858369346s" totalImagesPullingTime="16.035580017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:45:44.858127212 +0000 UTC m=+22.657600736" watchObservedRunningTime="2026-07-15 10:45:44.858369346 +0000 UTC m=+22.657842870"
Jul 15 10:45:44.880458 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:44.880425    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nq8j4" podStartSLOduration=2.600669778 podStartE2EDuration="22.880413552s" totalImagesPullingTime="20.279743774s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:45:44.88020232 +0000 UTC m=+22.679675846" watchObservedRunningTime="2026-07-15 10:45:44.880413552 +0000 UTC m=+22.679887077"
Jul 15 10:45:45.589459 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:45.589428    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:45.589578 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:45.589468    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:45.589578 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:45.589537    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:45.589578 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:45.589535    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:45.589746 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:45.589678    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:45.589813 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:45.589768    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:45.737514 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:45.737487    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v" event={"ID":"ef2acd01-96a6-43ff-b327-da4636c43ccd","Type":"ContainerStarted","Data":"d73fdb1aee00d0926759d44be0f40daf6c7db17a1e9438d6394b17af643e5ed8"}
Jul 15 10:45:45.737819 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:45.737572    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v" event={"ID":"ef2acd01-96a6-43ff-b327-da4636c43ccd","Type":"ContainerStarted","Data":"703edc99d54f4ae2d0b31af752d11f56f01a6527ca27c1eae9c4960e9ae13548"}
Jul 15 10:45:45.740427 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:45.740400    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-slr8n" event={"ID":"dbc90a95-67fa-4fb4-b4f4-20549cbf6194","Type":"ContainerStarted","Data":"4d28c8f13fff90cdff5682225fcea1086def72944ad99d91ead06e23baed640e"}
Jul 15 10:45:45.819181 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:45.819153    2536 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 10:45:46.568060 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:46.567948    2536 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T10:45:45.819176914Z","UUID":"d4f1b182-c478-4ba8-94fb-7c4c89b48881","Handler":null,"Name":"","Endpoint":""}
Jul 15 10:45:46.570126 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:46.570014    2536 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 10:45:46.570126 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:46.570101    2536 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 10:45:46.743629 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:46.743591    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal" event={"ID":"f97328aeac47e4524015111ec9ef9b68","Type":"ContainerStarted","Data":"41ca7a545a69218214d034e82a19209a8d7b2c94d08c644d6234239c56f5ef2e"}
Jul 15 10:45:46.745515 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:46.745474    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x" event={"ID":"c29a5184-864a-4a82-99a7-308aa7edc863","Type":"ContainerStarted","Data":"877ec4a6b1c2ba0901fb7111297e8609f853b9bd8524502a8614ff4d8e853944"}
Jul 15 10:45:46.764852 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:46.764814    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-slr8n" podStartSLOduration=4.59762706 podStartE2EDuration="24.764800671s" totalImagesPullingTime="20.167173611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:45:45.817488984 +0000 UTC m=+23.616962500" watchObservedRunningTime="2026-07-15 10:45:46.764800671 +0000 UTC m=+24.564274195"
Jul 15 10:45:46.765674 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:46.765632    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-178.ec2.internal" podStartSLOduration=23.765622504 podStartE2EDuration="23.765622504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:45:46.764735081 +0000 UTC m=+24.564208607" watchObservedRunningTime="2026-07-15 10:45:46.765622504 +0000 UTC m=+24.565096069"
Jul 15 10:45:46.942687 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:46.942658    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:46.943324 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:46.943301    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:47.589414 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:47.589384    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:47.589640 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:47.589481    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:47.589640 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:47.589490    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:47.589640 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:47.589490    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:47.589640 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:47.589570    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:47.589771 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:47.589664    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:47.749603 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:47.749569    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x" event={"ID":"c29a5184-864a-4a82-99a7-308aa7edc863","Type":"ContainerStarted","Data":"f91ede5c6280441d677a4d98580ddecef69b1baf49f56236e005443cfe2da8b6"}
Jul 15 10:45:47.753136 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:47.753091    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v" event={"ID":"ef2acd01-96a6-43ff-b327-da4636c43ccd","Type":"ContainerStarted","Data":"f2668a7c8e4147c875a627666130e6788e2453dd4396d84fb69e42fdf1a9b51b"}
Jul 15 10:45:47.753298 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:47.753174    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:47.753751 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:47.753731    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rbbsd"
Jul 15 10:45:47.793179 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:47.792254    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h286x" podStartSLOduration=3.512045064 podStartE2EDuration="25.792235237s" totalImagesPullingTime="22.280190173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:45:47.773371012 +0000 UTC m=+25.572844537" watchObservedRunningTime="2026-07-15 10:45:47.792235237 +0000 UTC m=+25.591708761"
Jul 15 10:45:49.589756 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:49.589588    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:49.590375 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:49.589589    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:49.590375 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:49.589841    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:49.590375 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:49.589929    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:49.590375 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:49.589588    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:49.590375 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:49.590030    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:49.761174 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:49.761143    2536 generic.go:356] "Generic (PLEG): container finished" podID="7950898a-8257-4b97-9fed-66ceafd63b56" containerID="3095109f28dc5860796526d78cf66554dfaa3748efd0c68b2ef818d188bdad4f" exitCode=0
Jul 15 10:45:49.761315 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:49.761223    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjftd" event={"ID":"7950898a-8257-4b97-9fed-66ceafd63b56","Type":"ContainerDied","Data":"3095109f28dc5860796526d78cf66554dfaa3748efd0c68b2ef818d188bdad4f"}
Jul 15 10:45:49.765243 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:49.765219    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v" event={"ID":"ef2acd01-96a6-43ff-b327-da4636c43ccd","Type":"ContainerStarted","Data":"1fc7ffc671e95ce69915af00a91930402bd0b3c5190e76f386bd5c838c49e4c9"}
Jul 15 10:45:49.817979 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:49.817942    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v" podStartSLOduration=7.461461077 podStartE2EDuration="27.817929297s" totalImagesPullingTime="20.35646822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:45:49.814467622 +0000 UTC m=+27.613941146" watchObservedRunningTime="2026-07-15 10:45:49.817929297 +0000 UTC m=+27.617402822"
Jul 15 10:45:50.767416 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:50.767391    2536 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 15 10:45:50.767977 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:50.767896    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:50.768041 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:50.767978    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:50.783626 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:50.783600    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:50.783800 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:50.783773    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:51.313400 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:51.313187    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cq9p2"]
Jul 15 10:45:51.313574 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:51.313486    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:51.313621 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:51.313569    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:51.317714 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:51.317684    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n9fpd"]
Jul 15 10:45:51.317852 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:51.317825    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:51.317949 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:51.317923    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:51.318301 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:51.318261    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6tndv"]
Jul 15 10:45:51.318387 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:51.318378    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:51.318467 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:51.318450    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:51.770664 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:51.770636    2536 generic.go:356] "Generic (PLEG): container finished" podID="7950898a-8257-4b97-9fed-66ceafd63b56" containerID="ea7aeffe047aacdd7330da2eea7d32c6a94b7bc1a57b4bcd270e62ea1c3ca20c" exitCode=0
Jul 15 10:45:51.771064 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:51.770718    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjftd" event={"ID":"7950898a-8257-4b97-9fed-66ceafd63b56","Type":"ContainerDied","Data":"ea7aeffe047aacdd7330da2eea7d32c6a94b7bc1a57b4bcd270e62ea1c3ca20c"}
Jul 15 10:45:51.771064 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:51.770831    2536 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 15 10:45:52.589749 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:52.589724    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:52.589928 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:52.589845    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:52.589974 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:52.589924    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:52.590021 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:52.590004    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:52.772702 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:52.772678    2536 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 15 10:45:53.170351 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:53.170247    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:45:53.589230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:53.589207    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:53.589383 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:53.589313    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:53.777394 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:53.777363    2536 generic.go:356] "Generic (PLEG): container finished" podID="7950898a-8257-4b97-9fed-66ceafd63b56" containerID="56430f10357e86d0a23aadfb9ca83b3095129d873c8d05d16e8d2496a6bd6c76" exitCode=0
Jul 15 10:45:53.777785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:53.777452    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjftd" event={"ID":"7950898a-8257-4b97-9fed-66ceafd63b56","Type":"ContainerDied","Data":"56430f10357e86d0a23aadfb9ca83b3095129d873c8d05d16e8d2496a6bd6c76"}
Jul 15 10:45:54.589717 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:54.589584    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:54.589892 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:54.589721    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:54.589892 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:54.589775    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:54.589963 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:54.589887    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:55.371841 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:55.371810    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:55.372245 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:55.371851    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:55.372245 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:55.371968    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:55.372245 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:55.371978    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:55.372245 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:55.372033    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret podName:f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:27.372017937 +0000 UTC m=+65.171491440 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret") pod "global-pull-secret-syncer-cq9p2" (UID: "f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5") : object "kube-system"/"original-pull-secret" not registered
Jul 15 10:45:55.372245 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:55.372046    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs podName:4f99bc36-1c09-41cc-9960-e8cf593e3e9c nodeName:}" failed. No retries permitted until 2026-07-15 10:46:27.372039928 +0000 UTC m=+65.171513431 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs") pod "network-metrics-daemon-6tndv" (UID: "4f99bc36-1c09-41cc-9960-e8cf593e3e9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 10:45:55.472668 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:55.472642    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxj9\" (UniqueName: \"kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9\") pod \"network-check-target-n9fpd\" (UID: \"28145093-25ab-4dd4-9ce2-edf6ea14ca27\") " pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:55.472852 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:55.472831    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 10:45:55.472901 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:55.472860    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 10:45:55.472901 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:55.472874    2536 projected.go:196] Error preparing data for projected volume kube-api-access-jmxj9 for pod openshift-network-diagnostics/network-check-target-n9fpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:55.472985 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:55.472931    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9 podName:28145093-25ab-4dd4-9ce2-edf6ea14ca27 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:27.472917479 +0000 UTC m=+65.272390984 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jmxj9" (UniqueName: "kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9") pod "network-check-target-n9fpd" (UID: "28145093-25ab-4dd4-9ce2-edf6ea14ca27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 10:45:55.590015 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:55.589939    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:55.590157 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:55.590043    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cq9p2" podUID="f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5"
Jul 15 10:45:56.589034 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:56.589002    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:56.589499 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:56.589126    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n9fpd" podUID="28145093-25ab-4dd4-9ce2-edf6ea14ca27"
Jul 15 10:45:56.589499 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:56.589212    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:56.589499 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:56.589361    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6tndv" podUID="4f99bc36-1c09-41cc-9960-e8cf593e3e9c"
Jul 15 10:45:57.038363 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.038331    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-178.ec2.internal" event="NodeReady"
Jul 15 10:45:57.038534 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.038461    2536 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 10:45:57.074686 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.074656    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d85c67bb6-hlzhc"]
Jul 15 10:45:57.081022 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.080992    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.084119 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.083902    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 10:45:57.084119 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.083949    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 10:45:57.084312 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.084228    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mqg6r\""
Jul 15 10:45:57.084547 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.084525    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 10:45:57.085544 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.085520    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.085671 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.085560    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f252f770-c18d-422d-b550-4157655338ba-ca-trust-extracted\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.085671 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.085590    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-trusted-ca\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.085787 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.085668    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-installation-pull-secrets\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.085787 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.085691    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-bound-sa-token\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.085787 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.085716    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnsdv\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-kube-api-access-tnsdv\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.085787 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.085765    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-registry-certificates\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.085997 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.085874    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-image-registry-private-configuration\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.090308 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.090268    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 10:45:57.090842 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.090820    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kbdbx"]
Jul 15 10:45:57.093046 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.093025    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kbdbx"
Jul 15 10:45:57.094971 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.094948    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d85c67bb6-hlzhc"]
Jul 15 10:45:57.095386 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.095361    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 10:45:57.095386 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.095374    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s2xwp\""
Jul 15 10:45:57.095646 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.095626    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-52trx"]
Jul 15 10:45:57.096021 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.095997    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 10:45:57.096477 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.096457    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 10:45:57.097955 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.097938    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.101744 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.101727    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 10:45:57.101834 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.101768    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8cmqw\""
Jul 15 10:45:57.101834 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.101776    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 10:45:57.102973 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.102951    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kbdbx"]
Jul 15 10:45:57.108098 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.108079    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-52trx"]
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186259    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-image-registry-private-configuration\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186333    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b2f4d3f-9e4f-470d-88ab-59c274848f26-config-volume\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186387    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186415    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f252f770-c18d-422d-b550-4157655338ba-ca-trust-extracted\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186445    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-trusted-ca\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186473    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c608745-d6fc-4ed0-9916-9f777692890c-cert\") pod \"ingress-canary-kbdbx\" (UID: \"1c608745-d6fc-4ed0-9916-9f777692890c\") " pod="openshift-ingress-canary/ingress-canary-kbdbx"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186496    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjxm7\" (UniqueName: \"kubernetes.io/projected/1c608745-d6fc-4ed0-9916-9f777692890c-kube-api-access-fjxm7\") pod \"ingress-canary-kbdbx\" (UID: \"1c608745-d6fc-4ed0-9916-9f777692890c\") " pod="openshift-ingress-canary/ingress-canary-kbdbx"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186522    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq9kx\" (UniqueName: \"kubernetes.io/projected/3b2f4d3f-9e4f-470d-88ab-59c274848f26-kube-api-access-mq9kx\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186549    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-installation-pull-secrets\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186573    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-bound-sa-token\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186596    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnsdv\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-kube-api-access-tnsdv\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186622    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-registry-certificates\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186650    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.186676    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b2f4d3f-9e4f-470d-88ab-59c274848f26-tmp-dir\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:57.186500    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:57.186760    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d85c67bb6-hlzhc: secret "image-registry-tls" not found
Jul 15 10:45:57.187481 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:57.186813    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls podName:f252f770-c18d-422d-b550-4157655338ba nodeName:}" failed. No retries permitted until 2026-07-15 10:45:57.686793643 +0000 UTC m=+35.486267150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls") pod "image-registry-7d85c67bb6-hlzhc" (UID: "f252f770-c18d-422d-b550-4157655338ba") : secret "image-registry-tls" not found
Jul 15 10:45:57.188298 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.187377    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-trusted-ca\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.188298 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.187448    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-registry-certificates\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.188298 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.187602    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f252f770-c18d-422d-b550-4157655338ba-ca-trust-extracted\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.191404 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.191351    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-installation-pull-secrets\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.191404 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.191353    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-image-registry-private-configuration\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.197848 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.197827    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnsdv\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-kube-api-access-tnsdv\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.198717 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.198698    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-bound-sa-token\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.287926 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.287894    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c608745-d6fc-4ed0-9916-9f777692890c-cert\") pod \"ingress-canary-kbdbx\" (UID: \"1c608745-d6fc-4ed0-9916-9f777692890c\") " pod="openshift-ingress-canary/ingress-canary-kbdbx"
Jul 15 10:45:57.288086 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.287935    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjxm7\" (UniqueName: \"kubernetes.io/projected/1c608745-d6fc-4ed0-9916-9f777692890c-kube-api-access-fjxm7\") pod \"ingress-canary-kbdbx\" (UID: \"1c608745-d6fc-4ed0-9916-9f777692890c\") " pod="openshift-ingress-canary/ingress-canary-kbdbx"
Jul 15 10:45:57.288086 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.287959    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq9kx\" (UniqueName: \"kubernetes.io/projected/3b2f4d3f-9e4f-470d-88ab-59c274848f26-kube-api-access-mq9kx\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.288086 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.287993    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.288086 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.288019    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b2f4d3f-9e4f-470d-88ab-59c274848f26-tmp-dir\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.288086 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.288077    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b2f4d3f-9e4f-470d-88ab-59c274848f26-config-volume\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.288421 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:57.288337    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 10:45:57.288421 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:57.288401    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls podName:3b2f4d3f-9e4f-470d-88ab-59c274848f26 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:57.788380502 +0000 UTC m=+35.587854024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls") pod "dns-default-52trx" (UID: "3b2f4d3f-9e4f-470d-88ab-59c274848f26") : secret "dns-default-metrics-tls" not found
Jul 15 10:45:57.288585 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.288559    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b2f4d3f-9e4f-470d-88ab-59c274848f26-tmp-dir\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.288664 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.288646    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3b2f4d3f-9e4f-470d-88ab-59c274848f26-config-volume\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.290690 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.290670    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c608745-d6fc-4ed0-9916-9f777692890c-cert\") pod \"ingress-canary-kbdbx\" (UID: \"1c608745-d6fc-4ed0-9916-9f777692890c\") " pod="openshift-ingress-canary/ingress-canary-kbdbx"
Jul 15 10:45:57.296897 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.296873    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq9kx\" (UniqueName: \"kubernetes.io/projected/3b2f4d3f-9e4f-470d-88ab-59c274848f26-kube-api-access-mq9kx\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.297390 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.297368    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjxm7\" (UniqueName: \"kubernetes.io/projected/1c608745-d6fc-4ed0-9916-9f777692890c-kube-api-access-fjxm7\") pod \"ingress-canary-kbdbx\" (UID: \"1c608745-d6fc-4ed0-9916-9f777692890c\") " pod="openshift-ingress-canary/ingress-canary-kbdbx"
Jul 15 10:45:57.405037 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.405010    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kbdbx"
Jul 15 10:45:57.589493 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.589259    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:45:57.591876 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.591855    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 10:45:57.691998 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.691963    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:57.692150 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:57.692117    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 10:45:57.692150 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:57.692133    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d85c67bb6-hlzhc: secret "image-registry-tls" not found
Jul 15 10:45:57.692236 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:57.692188    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls podName:f252f770-c18d-422d-b550-4157655338ba nodeName:}" failed. No retries permitted until 2026-07-15 10:45:58.692170604 +0000 UTC m=+36.491644116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls") pod "image-registry-7d85c67bb6-hlzhc" (UID: "f252f770-c18d-422d-b550-4157655338ba") : secret "image-registry-tls" not found
Jul 15 10:45:57.793339 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:57.793301    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:57.793502 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:57.793467    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 10:45:57.793568 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:57.793533    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls podName:3b2f4d3f-9e4f-470d-88ab-59c274848f26 nodeName:}" failed. No retries permitted until 2026-07-15 10:45:58.793512202 +0000 UTC m=+36.592985704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls") pod "dns-default-52trx" (UID: "3b2f4d3f-9e4f-470d-88ab-59c274848f26") : secret "dns-default-metrics-tls" not found
Jul 15 10:45:58.003214 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.003191    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kbdbx"]
Jul 15 10:45:58.006487 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:45:58.006462    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c608745_d6fc_4ed0_9916_9f777692890c.slice/crio-0b422d86ce6f2e16f712385cd2539db4f43e41e08159ff60adafc523316d7b2f WatchSource:0}: Error finding container 0b422d86ce6f2e16f712385cd2539db4f43e41e08159ff60adafc523316d7b2f: Status 404 returned error can't find the container with id 0b422d86ce6f2e16f712385cd2539db4f43e41e08159ff60adafc523316d7b2f
Jul 15 10:45:58.593150 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.593120    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:45:58.593804 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.593286    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:45:58.596803 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.596627    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 10:45:58.596803 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.596664    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 10:45:58.596803 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.596800    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfsm8\""
Jul 15 10:45:58.597120 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.597097    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xhf7j\""
Jul 15 10:45:58.597688 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.597543    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 10:45:58.700580 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.700551    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:45:58.700757 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:58.700685    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 10:45:58.700757 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:58.700700    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d85c67bb6-hlzhc: secret "image-registry-tls" not found
Jul 15 10:45:58.700868 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:58.700773    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls podName:f252f770-c18d-422d-b550-4157655338ba nodeName:}" failed. No retries permitted until 2026-07-15 10:46:00.700754213 +0000 UTC m=+38.500227716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls") pod "image-registry-7d85c67bb6-hlzhc" (UID: "f252f770-c18d-422d-b550-4157655338ba") : secret "image-registry-tls" not found
Jul 15 10:45:58.790687 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.790649    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kbdbx" event={"ID":"1c608745-d6fc-4ed0-9916-9f777692890c","Type":"ContainerStarted","Data":"0b422d86ce6f2e16f712385cd2539db4f43e41e08159ff60adafc523316d7b2f"}
Jul 15 10:45:58.793972 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.793942    2536 generic.go:356] "Generic (PLEG): container finished" podID="7950898a-8257-4b97-9fed-66ceafd63b56" containerID="744e1dc6ad5dc350cfb216f8aa628d7f9d50889aad20d133acca9e19dc997a27" exitCode=0
Jul 15 10:45:58.794078 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.793983    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjftd" event={"ID":"7950898a-8257-4b97-9fed-66ceafd63b56","Type":"ContainerDied","Data":"744e1dc6ad5dc350cfb216f8aa628d7f9d50889aad20d133acca9e19dc997a27"}
Jul 15 10:45:58.801157 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:58.800907    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:45:58.801157 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:58.801150    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 10:45:58.801294 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:45:58.801215    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls podName:3b2f4d3f-9e4f-470d-88ab-59c274848f26 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:00.801193662 +0000 UTC m=+38.600667165 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls") pod "dns-default-52trx" (UID: "3b2f4d3f-9e4f-470d-88ab-59c274848f26") : secret "dns-default-metrics-tls" not found
Jul 15 10:45:59.800388 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:59.800165    2536 generic.go:356] "Generic (PLEG): container finished" podID="7950898a-8257-4b97-9fed-66ceafd63b56" containerID="a1814fe3f8af6fb497884aef419827737cafaffad62eee8499e2e850861046a9" exitCode=0
Jul 15 10:45:59.800853 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:45:59.800253    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjftd" event={"ID":"7950898a-8257-4b97-9fed-66ceafd63b56","Type":"ContainerDied","Data":"a1814fe3f8af6fb497884aef419827737cafaffad62eee8499e2e850861046a9"}
Jul 15 10:46:00.717673 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:00.717646    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:46:00.717823 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:00.717750    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 10:46:00.717823 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:00.717762    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d85c67bb6-hlzhc: secret "image-registry-tls" not found
Jul 15 10:46:00.717823 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:00.717812    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls podName:f252f770-c18d-422d-b550-4157655338ba nodeName:}" failed. No retries permitted until 2026-07-15 10:46:04.717797718 +0000 UTC m=+42.517271220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls") pod "image-registry-7d85c67bb6-hlzhc" (UID: "f252f770-c18d-422d-b550-4157655338ba") : secret "image-registry-tls" not found
Jul 15 10:46:00.804030 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:00.804000    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kbdbx" event={"ID":"1c608745-d6fc-4ed0-9916-9f777692890c","Type":"ContainerStarted","Data":"017d9c0ef6294a150a89771e403e0d348cea497e205f75b34c3c9db4c17c6d4d"}
Jul 15 10:46:00.807166 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:00.807136    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjftd" event={"ID":"7950898a-8257-4b97-9fed-66ceafd63b56","Type":"ContainerStarted","Data":"e27076c3c379ef7d6b8b4cec5eb2c545defd9aa959d1615311c572e9994195e4"}
Jul 15 10:46:00.818581 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:00.818563    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:46:00.818728 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:00.818713    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 10:46:00.818785 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:00.818774    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls podName:3b2f4d3f-9e4f-470d-88ab-59c274848f26 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:04.818758767 +0000 UTC m=+42.618232288 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls") pod "dns-default-52trx" (UID: "3b2f4d3f-9e4f-470d-88ab-59c274848f26") : secret "dns-default-metrics-tls" not found
Jul 15 10:46:00.825508 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:00.825468    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kbdbx" podStartSLOduration=1.4013795820000001 podStartE2EDuration="3.825455712s" totalImagesPullingTime="2.42407613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:45:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:46:00.824961484 +0000 UTC m=+38.624435008" watchObservedRunningTime="2026-07-15 10:46:00.825455712 +0000 UTC m=+38.624929227"
Jul 15 10:46:00.849091 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:00.849043    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rjftd" podStartSLOduration=7.834603772 podStartE2EDuration="38.849026494s" totalImagesPullingTime="31.014422722s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:46:00.847945827 +0000 UTC m=+38.647419365" watchObservedRunningTime="2026-07-15 10:46:00.849026494 +0000 UTC m=+38.648500021"
Jul 15 10:46:04.745918 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:04.745887    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:46:04.746233 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:04.745994    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 10:46:04.746233 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:04.746005    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d85c67bb6-hlzhc: secret "image-registry-tls" not found
Jul 15 10:46:04.746233 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:04.746051    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls podName:f252f770-c18d-422d-b550-4157655338ba nodeName:}" failed. No retries permitted until 2026-07-15 10:46:12.746037915 +0000 UTC m=+50.545511417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls") pod "image-registry-7d85c67bb6-hlzhc" (UID: "f252f770-c18d-422d-b550-4157655338ba") : secret "image-registry-tls" not found
Jul 15 10:46:04.846118 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:04.846092    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:46:04.846243 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:04.846227    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 10:46:04.846313 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:04.846295    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls podName:3b2f4d3f-9e4f-470d-88ab-59c274848f26 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:12.846267175 +0000 UTC m=+50.645740677 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls") pod "dns-default-52trx" (UID: "3b2f4d3f-9e4f-470d-88ab-59c274848f26") : secret "dns-default-metrics-tls" not found
Jul 15 10:46:12.802669 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:12.802640    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:46:12.803099 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:12.802742    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 10:46:12.803099 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:12.802752    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d85c67bb6-hlzhc: secret "image-registry-tls" not found
Jul 15 10:46:12.803099 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:12.802806    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls podName:f252f770-c18d-422d-b550-4157655338ba nodeName:}" failed. No retries permitted until 2026-07-15 10:46:28.802792375 +0000 UTC m=+66.602265877 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls") pod "image-registry-7d85c67bb6-hlzhc" (UID: "f252f770-c18d-422d-b550-4157655338ba") : secret "image-registry-tls" not found
Jul 15 10:46:12.903470 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:12.903440    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:46:12.903596 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:12.903532    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 10:46:12.903596 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:12.903577    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls podName:3b2f4d3f-9e4f-470d-88ab-59c274848f26 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:28.903565292 +0000 UTC m=+66.703038798 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls") pod "dns-default-52trx" (UID: "3b2f4d3f-9e4f-470d-88ab-59c274848f26") : secret "dns-default-metrics-tls" not found
Jul 15 10:46:23.788180 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:23.788155    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kbz5v"
Jul 15 10:46:27.399682 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.399637    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:46:27.399682 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.399683    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:46:27.402495 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.402472    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 10:46:27.402612 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.402594    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 10:46:27.410417 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:27.410400    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 15 10:46:27.410500 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:27.410453    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs podName:4f99bc36-1c09-41cc-9960-e8cf593e3e9c nodeName:}" failed. No retries permitted until 2026-07-15 10:47:31.410436818 +0000 UTC m=+129.209910325 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs") pod "network-metrics-daemon-6tndv" (UID: "4f99bc36-1c09-41cc-9960-e8cf593e3e9c") : secret "metrics-daemon-secret" not found
Jul 15 10:46:27.414020 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.413997    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5-original-pull-secret\") pod \"global-pull-secret-syncer-cq9p2\" (UID: \"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5\") " pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:46:27.500648 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.500613    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxj9\" (UniqueName: \"kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9\") pod \"network-check-target-n9fpd\" (UID: \"28145093-25ab-4dd4-9ce2-edf6ea14ca27\") " pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:46:27.503360 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.503342    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 10:46:27.513606 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.513584    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 10:46:27.524700 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.524676    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmxj9\" (UniqueName: \"kubernetes.io/projected/28145093-25ab-4dd4-9ce2-edf6ea14ca27-kube-api-access-jmxj9\") pod \"network-check-target-n9fpd\" (UID: \"28145093-25ab-4dd4-9ce2-edf6ea14ca27\") " pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:46:27.600265 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.600227    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cq9p2"
Jul 15 10:46:27.712555 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.712528    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xhf7j\""
Jul 15 10:46:27.720559 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.720542    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:46:27.730965 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.730944    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cq9p2"]
Jul 15 10:46:27.731730 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:46:27.731704    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf40144cc_a5a5_47a0_8e2a_8de3ddde2ad5.slice/crio-d4ca98fa82aa6e883279bdea58ad1232691ed522933af25f2715510e5f0ee5a8 WatchSource:0}: Error finding container d4ca98fa82aa6e883279bdea58ad1232691ed522933af25f2715510e5f0ee5a8: Status 404 returned error can't find the container with id d4ca98fa82aa6e883279bdea58ad1232691ed522933af25f2715510e5f0ee5a8
Jul 15 10:46:27.851456 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.851418    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n9fpd"]
Jul 15 10:46:27.854721 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:46:27.854687    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28145093_25ab_4dd4_9ce2_edf6ea14ca27.slice/crio-67d66addbb7c687be311cc7c692df69d4f20fa23a94e3c96f93a8a9ebd1ecbf5 WatchSource:0}: Error finding container 67d66addbb7c687be311cc7c692df69d4f20fa23a94e3c96f93a8a9ebd1ecbf5: Status 404 returned error can't find the container with id 67d66addbb7c687be311cc7c692df69d4f20fa23a94e3c96f93a8a9ebd1ecbf5
Jul 15 10:46:27.868810 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.868784    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cq9p2" event={"ID":"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5","Type":"ContainerStarted","Data":"d4ca98fa82aa6e883279bdea58ad1232691ed522933af25f2715510e5f0ee5a8"}
Jul 15 10:46:27.869792 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:27.869772    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n9fpd" event={"ID":"28145093-25ab-4dd4-9ce2-edf6ea14ca27","Type":"ContainerStarted","Data":"67d66addbb7c687be311cc7c692df69d4f20fa23a94e3c96f93a8a9ebd1ecbf5"}
Jul 15 10:46:28.088952 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.088875    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-tcwtw"]
Jul 15 10:46:28.092787 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.092770    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.096769 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.095292    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-s8hsq\""
Jul 15 10:46:28.096769 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.095588    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 15 10:46:28.096769 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.095812    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 15 10:46:28.097794 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.097746    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 10:46:28.097794 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.097770    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 10:46:28.102459 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.102442    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 15 10:46:28.113528 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.113507    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-tcwtw"]
Jul 15 10:46:28.206629 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.206593    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.206809 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.206644    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-tmp\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.206809 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.206726    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-snapshots\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.206911 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.206827    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-serving-cert\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.206911 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.206881    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5dc4\" (UniqueName: \"kubernetes.io/projected/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-kube-api-access-h5dc4\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.207004 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.206917    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.274509 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.273544    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"]
Jul 15 10:46:28.276624 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.276570    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:28.280559 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.280536    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 15 10:46:28.280773 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.280757    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 10:46:28.280964 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.280948    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 10:46:28.281147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.281133    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-tbtb5\""
Jul 15 10:46:28.281664 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.281642    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 15 10:46:28.289716 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.289695    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"]
Jul 15 10:46:28.307425 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.307399    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5dc4\" (UniqueName: \"kubernetes.io/projected/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-kube-api-access-h5dc4\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.307542 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.307454    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.307542 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.307487    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.307542 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.307513    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-tmp\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.307542 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.307540    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-snapshots\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.307929 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.307755    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-serving-cert\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.308036 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.307946    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-tmp\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.308189 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.308173    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-snapshots\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.308848 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.308825    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.308943 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.308935    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.310939 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.310915    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-serving-cert\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.320639 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.320615    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5dc4\" (UniqueName: \"kubernetes.io/projected/c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9-kube-api-access-h5dc4\") pod \"insights-operator-68d9dd94cf-tcwtw\" (UID: \"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9\") " pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.403098 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.403022    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw"
Jul 15 10:46:28.408615 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.408589    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c9493868-2500-412f-bfcc-f02972b330a3-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:28.408735 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.408637    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:28.408802 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.408731    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh2nx\" (UniqueName: \"kubernetes.io/projected/c9493868-2500-412f-bfcc-f02972b330a3-kube-api-access-hh2nx\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:28.509862 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.509828    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh2nx\" (UniqueName: \"kubernetes.io/projected/c9493868-2500-412f-bfcc-f02972b330a3-kube-api-access-hh2nx\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:28.510006 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.509883    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c9493868-2500-412f-bfcc-f02972b330a3-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:28.510101 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.510074    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:28.510317 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:28.510293    2536 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 15 10:46:28.510447 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:28.510374    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls podName:c9493868-2500-412f-bfcc-f02972b330a3 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:29.010352093 +0000 UTC m=+66.809825602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-xn2p6" (UID: "c9493868-2500-412f-bfcc-f02972b330a3") : secret "cluster-monitoring-operator-tls" not found
Jul 15 10:46:28.510609 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.510588    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c9493868-2500-412f-bfcc-f02972b330a3-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:28.522545 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.522515    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh2nx\" (UniqueName: \"kubernetes.io/projected/c9493868-2500-412f-bfcc-f02972b330a3-kube-api-access-hh2nx\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:28.560341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.560301    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-tcwtw"]
Jul 15 10:46:28.565742 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:46:28.565709    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7c27c79_058c_4b3b_a5f4_c72f0bf2b2d9.slice/crio-bbd437e2f89ca1b9b5b7d3a313ebe76298cad6e3448469ac28a595faf0f26787 WatchSource:0}: Error finding container bbd437e2f89ca1b9b5b7d3a313ebe76298cad6e3448469ac28a595faf0f26787: Status 404 returned error can't find the container with id bbd437e2f89ca1b9b5b7d3a313ebe76298cad6e3448469ac28a595faf0f26787
Jul 15 10:46:28.813239 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.813205    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:46:28.813461 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:28.813422    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 10:46:28.813461 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:28.813444    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d85c67bb6-hlzhc: secret "image-registry-tls" not found
Jul 15 10:46:28.813573 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:28.813525    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls podName:f252f770-c18d-422d-b550-4157655338ba nodeName:}" failed. No retries permitted until 2026-07-15 10:47:00.813503447 +0000 UTC m=+98.612976963 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls") pod "image-registry-7d85c67bb6-hlzhc" (UID: "f252f770-c18d-422d-b550-4157655338ba") : secret "image-registry-tls" not found
Jul 15 10:46:28.874249 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.874206    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw" event={"ID":"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9","Type":"ContainerStarted","Data":"bbd437e2f89ca1b9b5b7d3a313ebe76298cad6e3448469ac28a595faf0f26787"}
Jul 15 10:46:28.914443 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:28.914400    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:46:28.914590 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:28.914570    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 10:46:28.914649 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:28.914637    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls podName:3b2f4d3f-9e4f-470d-88ab-59c274848f26 nodeName:}" failed. No retries permitted until 2026-07-15 10:47:00.914617587 +0000 UTC m=+98.714091092 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls") pod "dns-default-52trx" (UID: "3b2f4d3f-9e4f-470d-88ab-59c274848f26") : secret "dns-default-metrics-tls" not found
Jul 15 10:46:29.015724 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:29.015686    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:29.015909 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:29.015841    2536 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 15 10:46:29.015909 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:29.015902    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls podName:c9493868-2500-412f-bfcc-f02972b330a3 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:30.015885073 +0000 UTC m=+67.815358578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-xn2p6" (UID: "c9493868-2500-412f-bfcc-f02972b330a3") : secret "cluster-monitoring-operator-tls" not found
Jul 15 10:46:30.026083 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:30.025470    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:30.026083 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:30.025649    2536 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 15 10:46:30.026083 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:30.025726    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls podName:c9493868-2500-412f-bfcc-f02972b330a3 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:32.025706109 +0000 UTC m=+69.825179615 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-xn2p6" (UID: "c9493868-2500-412f-bfcc-f02972b330a3") : secret "cluster-monitoring-operator-tls" not found
Jul 15 10:46:32.044409 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:32.044362    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:32.044827 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:32.044543    2536 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 15 10:46:32.044827 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:32.044612    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls podName:c9493868-2500-412f-bfcc-f02972b330a3 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:36.044595455 +0000 UTC m=+73.844068958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-xn2p6" (UID: "c9493868-2500-412f-bfcc-f02972b330a3") : secret "cluster-monitoring-operator-tls" not found
Jul 15 10:46:33.889887 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:33.889853    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw" event={"ID":"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9","Type":"ContainerStarted","Data":"116e90edb65d3e056f669b85222095bee93f93cd33b2591a8dfb315895a592aa"}
Jul 15 10:46:33.891257 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:33.891231    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cq9p2" event={"ID":"f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5","Type":"ContainerStarted","Data":"45f8a02743b4dabdf5f1ae4b309650d9631b2ec97aa812b75c733d34a12cd97b"}
Jul 15 10:46:33.892595 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:33.892575    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n9fpd" event={"ID":"28145093-25ab-4dd4-9ce2-edf6ea14ca27","Type":"ContainerStarted","Data":"0d77dd22d8801cf041049ded1a6ee010aab01f7eb836b8fdc0178094aa53b136"}
Jul 15 10:46:33.892710 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:33.892699    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:46:33.912594 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:33.912553    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw" podStartSLOduration=1.3666776440000001 podStartE2EDuration="5.912539867s" totalImagesPullingTime="4.545862223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:46:33.912259822 +0000 UTC m=+71.711733348" watchObservedRunningTime="2026-07-15 10:46:33.912539867 +0000 UTC m=+71.712013423"
Jul 15 10:46:33.950074 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:33.950019    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cq9p2" podStartSLOduration=66.560677557 podStartE2EDuration="1m11.950004392s" totalImagesPullingTime="5.389326835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:46:33.930330232 +0000 UTC m=+71.729803756" watchObservedRunningTime="2026-07-15 10:46:33.950004392 +0000 UTC m=+71.749477912"
Jul 15 10:46:33.950359 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:33.950336    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n9fpd" podStartSLOduration=66.694958747 podStartE2EDuration="1m11.950328844s" totalImagesPullingTime="5.255370097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:46:33.949567353 +0000 UTC m=+71.749040872" watchObservedRunningTime="2026-07-15 10:46:33.950328844 +0000 UTC m=+71.749802369"
Jul 15 10:46:34.121470 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.121439    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vlj6f"]
Jul 15 10:46:34.124682 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.124666    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.127335 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.127315    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 10:46:34.127688 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.127668    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zgdp7\""
Jul 15 10:46:34.127789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.127702    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 10:46:34.150249 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.150193    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vlj6f"]
Jul 15 10:46:34.262846 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.262820    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b144126b-086e-4cea-bc1a-3df5c25b8c35-data-volume\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.262985 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.262853    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.262985 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.262904    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b144126b-086e-4cea-bc1a-3df5c25b8c35-crio-socket\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.262985 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.262923    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttrc9\" (UniqueName: \"kubernetes.io/projected/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-api-access-ttrc9\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.262985 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.262955    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.363777 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.363736    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b144126b-086e-4cea-bc1a-3df5c25b8c35-crio-socket\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.363777 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.363782    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttrc9\" (UniqueName: \"kubernetes.io/projected/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-api-access-ttrc9\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.363993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.363815    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.363993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.363866    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b144126b-086e-4cea-bc1a-3df5c25b8c35-data-volume\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.363993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.363879    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b144126b-086e-4cea-bc1a-3df5c25b8c35-crio-socket\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.363993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.363893    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.363993 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:34.363953    2536 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 10:46:34.363993 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:34.363987    2536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 10:46:34.364202 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:34.364025    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm podName:b144126b-086e-4cea-bc1a-3df5c25b8c35 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:34.864006749 +0000 UTC m=+72.663480267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vlj6f" (UID: "b144126b-086e-4cea-bc1a-3df5c25b8c35") : configmap "kube-rbac-proxy" not found
Jul 15 10:46:34.364202 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:34.364052    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls podName:b144126b-086e-4cea-bc1a-3df5c25b8c35 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:34.864035156 +0000 UTC m=+72.663508675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlj6f" (UID: "b144126b-086e-4cea-bc1a-3df5c25b8c35") : secret "insights-runtime-extractor-tls" not found
Jul 15 10:46:34.364202 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.364188    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b144126b-086e-4cea-bc1a-3df5c25b8c35-data-volume\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.372708 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.372690    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttrc9\" (UniqueName: \"kubernetes.io/projected/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-api-access-ttrc9\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.866979 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.866952    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.867130 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:34.867001    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:34.867130 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:34.867095    2536 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 10:46:34.867130 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:34.867097    2536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 10:46:34.867229 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:34.867159    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls podName:b144126b-086e-4cea-bc1a-3df5c25b8c35 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:35.867144559 +0000 UTC m=+73.666618061 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlj6f" (UID: "b144126b-086e-4cea-bc1a-3df5c25b8c35") : secret "insights-runtime-extractor-tls" not found
Jul 15 10:46:34.867229 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:34.867173    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm podName:b144126b-086e-4cea-bc1a-3df5c25b8c35 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:35.867167126 +0000 UTC m=+73.666640629 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vlj6f" (UID: "b144126b-086e-4cea-bc1a-3df5c25b8c35") : configmap "kube-rbac-proxy" not found
Jul 15 10:46:35.874852 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:35.874817    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:35.875206 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:35.874868    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:35.875206 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:35.874957    2536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 10:46:35.875206 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:35.875020    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls podName:b144126b-086e-4cea-bc1a-3df5c25b8c35 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:37.87500612 +0000 UTC m=+75.674479623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlj6f" (UID: "b144126b-086e-4cea-bc1a-3df5c25b8c35") : secret "insights-runtime-extractor-tls" not found
Jul 15 10:46:35.875206 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:35.874960    2536 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 10:46:35.875206 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:35.875093    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm podName:b144126b-086e-4cea-bc1a-3df5c25b8c35 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:37.875081191 +0000 UTC m=+75.674554694 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vlj6f" (UID: "b144126b-086e-4cea-bc1a-3df5c25b8c35") : configmap "kube-rbac-proxy" not found
Jul 15 10:46:36.076543 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:36.076513    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:36.076673 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:36.076650    2536 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 15 10:46:36.076722 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:36.076711    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls podName:c9493868-2500-412f-bfcc-f02972b330a3 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:44.07669552 +0000 UTC m=+81.876169024 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-xn2p6" (UID: "c9493868-2500-412f-bfcc-f02972b330a3") : secret "cluster-monitoring-operator-tls" not found
Jul 15 10:46:36.888659 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:36.888630    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s986k_764e82f9-85c4-44bb-9a80-72c5e55a953b/dns-node-resolver/0.log"
Jul 15 10:46:37.691261 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:37.691228    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gx9mp_9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf/node-ca/0.log"
Jul 15 10:46:37.887228 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:37.887199    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:37.887391 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:37.887249    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:37.887391 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:37.887331    2536 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 10:46:37.887391 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:37.887353    2536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 10:46:37.887391 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:37.887388    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm podName:b144126b-086e-4cea-bc1a-3df5c25b8c35 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:41.887374402 +0000 UTC m=+79.686847905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vlj6f" (UID: "b144126b-086e-4cea-bc1a-3df5c25b8c35") : configmap "kube-rbac-proxy" not found
Jul 15 10:46:37.887541 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:37.887401    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls podName:b144126b-086e-4cea-bc1a-3df5c25b8c35 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:41.887395386 +0000 UTC m=+79.686868888 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlj6f" (UID: "b144126b-086e-4cea-bc1a-3df5c25b8c35") : secret "insights-runtime-extractor-tls" not found
Jul 15 10:46:38.488238 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:38.488210    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kbdbx_1c608745-d6fc-4ed0-9916-9f777692890c/serve-healthcheck-canary/0.log"
Jul 15 10:46:41.913189 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:41.913157    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:41.913611 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:41.913207    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:41.913611 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:41.913266    2536 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 10:46:41.913611 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:41.913347    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm podName:b144126b-086e-4cea-bc1a-3df5c25b8c35 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:49.913327614 +0000 UTC m=+87.712801120 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vlj6f" (UID: "b144126b-086e-4cea-bc1a-3df5c25b8c35") : configmap "kube-rbac-proxy" not found
Jul 15 10:46:41.913611 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:41.913403    2536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 10:46:41.913611 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:46:41.913457    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls podName:b144126b-086e-4cea-bc1a-3df5c25b8c35 nodeName:}" failed. No retries permitted until 2026-07-15 10:46:49.913443006 +0000 UTC m=+87.712916511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vlj6f" (UID: "b144126b-086e-4cea-bc1a-3df5c25b8c35") : secret "insights-runtime-extractor-tls" not found
Jul 15 10:46:44.128856 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:44.128744    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:44.131365 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:44.131344    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9493868-2500-412f-bfcc-f02972b330a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-xn2p6\" (UID: \"c9493868-2500-412f-bfcc-f02972b330a3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:44.188169 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:44.188144    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"
Jul 15 10:46:44.323286 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:44.323249    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6"]
Jul 15 10:46:44.332603 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:46:44.332575    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9493868_2500_412f_bfcc_f02972b330a3.slice/crio-a1e2c7ebe81995c7085f124780bdf0b41c40a20656b452586977a16a8e2239ea WatchSource:0}: Error finding container a1e2c7ebe81995c7085f124780bdf0b41c40a20656b452586977a16a8e2239ea: Status 404 returned error can't find the container with id a1e2c7ebe81995c7085f124780bdf0b41c40a20656b452586977a16a8e2239ea
Jul 15 10:46:44.919264 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:44.919232    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6" event={"ID":"c9493868-2500-412f-bfcc-f02972b330a3","Type":"ContainerStarted","Data":"a1e2c7ebe81995c7085f124780bdf0b41c40a20656b452586977a16a8e2239ea"}
Jul 15 10:46:46.925417 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:46.925383    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6" event={"ID":"c9493868-2500-412f-bfcc-f02972b330a3","Type":"ContainerStarted","Data":"e12f8c23392b2deab668af1f6dbc4477253f1bc0f8f4ec85df74376a8c2a7391"}
Jul 15 10:46:46.943246 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:46.943195    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-xn2p6" podStartSLOduration=16.949988507 podStartE2EDuration="18.943179194s" totalImagesPullingTime="1.993190687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:46:46.9431076 +0000 UTC m=+84.742581124" watchObservedRunningTime="2026-07-15 10:46:46.943179194 +0000 UTC m=+84.742652720"
Jul 15 10:46:49.973958 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:49.973925    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:49.974434 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:49.974024    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:49.974501 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:49.974441    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b144126b-086e-4cea-bc1a-3df5c25b8c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:49.976402 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:49.976382    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b144126b-086e-4cea-bc1a-3df5c25b8c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vlj6f\" (UID: \"b144126b-086e-4cea-bc1a-3df5c25b8c35\") " pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:50.034028 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:50.034004    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vlj6f"
Jul 15 10:46:50.161320 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:50.161212    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vlj6f"]
Jul 15 10:46:50.163742 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:46:50.163712    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb144126b_086e_4cea_bc1a_3df5c25b8c35.slice/crio-1f5a6238f123be824103c497d68032820d116cc7fab25d6ac2ed19f508462d77 WatchSource:0}: Error finding container 1f5a6238f123be824103c497d68032820d116cc7fab25d6ac2ed19f508462d77: Status 404 returned error can't find the container with id 1f5a6238f123be824103c497d68032820d116cc7fab25d6ac2ed19f508462d77
Jul 15 10:46:50.939095 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:50.939005    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlj6f" event={"ID":"b144126b-086e-4cea-bc1a-3df5c25b8c35","Type":"ContainerStarted","Data":"1f5a6238f123be824103c497d68032820d116cc7fab25d6ac2ed19f508462d77"}
Jul 15 10:46:55.958913 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:55.958866    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlj6f" event={"ID":"b144126b-086e-4cea-bc1a-3df5c25b8c35","Type":"ContainerStarted","Data":"bb5eb7e245f699e0c611c15b67b21e943617babfa5c46e9edeff5c5c3a744596"}
Jul 15 10:46:57.107354 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.107320    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7"]
Jul 15 10:46:57.117528 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.117504    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7"
Jul 15 10:46:57.152514 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.152481    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 10:46:57.152648 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.152552    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 15 10:46:57.152710 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.152659    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 10:46:57.153559 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.153532    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7"]
Jul 15 10:46:57.157572 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.157549    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 10:46:57.158898 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.158880    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-h2gdt\""
Jul 15 10:46:57.209539 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.209510    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-d68nc"]
Jul 15 10:46:57.214037 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.214017    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-d68nc"
Jul 15 10:46:57.216685 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.216664    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 15 10:46:57.218612 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.218589    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 15 10:46:57.218714 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.218694    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-h2p9n\""
Jul 15 10:46:57.233089 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.233068    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt8k5\" (UniqueName: \"kubernetes.io/projected/b8a46b16-e479-4666-b59a-96902941c826-kube-api-access-qt8k5\") pod \"managed-serviceaccount-addon-agent-74878b79d6-7h8p7\" (UID: \"b8a46b16-e479-4666-b59a-96902941c826\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7"
Jul 15 10:46:57.233202 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.233134    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8a46b16-e479-4666-b59a-96902941c826-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74878b79d6-7h8p7\" (UID: \"b8a46b16-e479-4666-b59a-96902941c826\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7"
Jul 15 10:46:57.240111 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.240084    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-d68nc"]
Jul 15 10:46:57.294885 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.294855    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-znd4t"]
Jul 15 10:46:57.299013 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.298995    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-znd4t"
Jul 15 10:46:57.301658 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.301637    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 10:46:57.301746 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.301686    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ng7wx\""
Jul 15 10:46:57.301746 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.301687    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 10:46:57.312947 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.312927    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-znd4t"]
Jul 15 10:46:57.333544 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.333516    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/470d651d-0c05-4629-a564-50a940da19d1-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-d68nc\" (UID: \"470d651d-0c05-4629-a564-50a940da19d1\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-d68nc"
Jul 15 10:46:57.333789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.333559    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt8k5\" (UniqueName: \"kubernetes.io/projected/b8a46b16-e479-4666-b59a-96902941c826-kube-api-access-qt8k5\") pod \"managed-serviceaccount-addon-agent-74878b79d6-7h8p7\" (UID: \"b8a46b16-e479-4666-b59a-96902941c826\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7"
Jul 15 10:46:57.333789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.333612    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/470d651d-0c05-4629-a564-50a940da19d1-nginx-conf\") pod \"networking-console-plugin-6cf856758c-d68nc\" (UID: \"470d651d-0c05-4629-a564-50a940da19d1\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-d68nc"
Jul 15 10:46:57.333789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.333664    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8a46b16-e479-4666-b59a-96902941c826-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74878b79d6-7h8p7\" (UID: \"b8a46b16-e479-4666-b59a-96902941c826\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7"
Jul 15 10:46:57.336615 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.336592    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8a46b16-e479-4666-b59a-96902941c826-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74878b79d6-7h8p7\" (UID: \"b8a46b16-e479-4666-b59a-96902941c826\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7"
Jul 15 10:46:57.347501 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.347482    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt8k5\" (UniqueName: \"kubernetes.io/projected/b8a46b16-e479-4666-b59a-96902941c826-kube-api-access-qt8k5\") pod \"managed-serviceaccount-addon-agent-74878b79d6-7h8p7\" (UID: \"b8a46b16-e479-4666-b59a-96902941c826\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7"
Jul 15 10:46:57.434550 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.434481    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7"
Jul 15 10:46:57.434697 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.434671    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/470d651d-0c05-4629-a564-50a940da19d1-nginx-conf\") pod \"networking-console-plugin-6cf856758c-d68nc\" (UID: \"470d651d-0c05-4629-a564-50a940da19d1\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-d68nc"
Jul 15 10:46:57.434775 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.434745    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r57cl\" (UniqueName: \"kubernetes.io/projected/bf9442c2-785b-449d-8cea-dee4b0564a08-kube-api-access-r57cl\") pod \"downloads-68c9cbb497-znd4t\" (UID: \"bf9442c2-785b-449d-8cea-dee4b0564a08\") " pod="openshift-console/downloads-68c9cbb497-znd4t"
Jul 15 10:46:57.434897 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.434833    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/470d651d-0c05-4629-a564-50a940da19d1-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-d68nc\" (UID: \"470d651d-0c05-4629-a564-50a940da19d1\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-d68nc"
Jul 15 10:46:57.435523 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.435426    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/470d651d-0c05-4629-a564-50a940da19d1-nginx-conf\") pod \"networking-console-plugin-6cf856758c-d68nc\" (UID: \"470d651d-0c05-4629-a564-50a940da19d1\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-d68nc"
Jul 15 10:46:57.437442 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.437420    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/470d651d-0c05-4629-a564-50a940da19d1-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-d68nc\" (UID: \"470d651d-0c05-4629-a564-50a940da19d1\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-d68nc"
Jul 15 10:46:57.524159 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.524122    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-d68nc"
Jul 15 10:46:57.535577 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.535545    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r57cl\" (UniqueName: \"kubernetes.io/projected/bf9442c2-785b-449d-8cea-dee4b0564a08-kube-api-access-r57cl\") pod \"downloads-68c9cbb497-znd4t\" (UID: \"bf9442c2-785b-449d-8cea-dee4b0564a08\") " pod="openshift-console/downloads-68c9cbb497-znd4t"
Jul 15 10:46:57.546007 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.545989    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r57cl\" (UniqueName: \"kubernetes.io/projected/bf9442c2-785b-449d-8cea-dee4b0564a08-kube-api-access-r57cl\") pod \"downloads-68c9cbb497-znd4t\" (UID: \"bf9442c2-785b-449d-8cea-dee4b0564a08\") " pod="openshift-console/downloads-68c9cbb497-znd4t"
Jul 15 10:46:57.609404 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:57.609373    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-znd4t"
Jul 15 10:46:58.810888 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:58.810852    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7"]
Jul 15 10:46:58.816498 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:46:58.816465    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8a46b16_e479_4666_b59a_96902941c826.slice/crio-2d144ee7d87cc5a1fb1db52dc9dfe4d57e8dbae168f29b81eec2900c518a869e WatchSource:0}: Error finding container 2d144ee7d87cc5a1fb1db52dc9dfe4d57e8dbae168f29b81eec2900c518a869e: Status 404 returned error can't find the container with id 2d144ee7d87cc5a1fb1db52dc9dfe4d57e8dbae168f29b81eec2900c518a869e
Jul 15 10:46:58.826763 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:58.826735    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-znd4t"]
Jul 15 10:46:58.829858 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:46:58.829821    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9442c2_785b_449d_8cea_dee4b0564a08.slice/crio-495c1e453cb713ad8d40c7d33c1a5d9fd92049110a092a1f889fe435141240c0 WatchSource:0}: Error finding container 495c1e453cb713ad8d40c7d33c1a5d9fd92049110a092a1f889fe435141240c0: Status 404 returned error can't find the container with id 495c1e453cb713ad8d40c7d33c1a5d9fd92049110a092a1f889fe435141240c0
Jul 15 10:46:58.840749 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:58.840725    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-d68nc"]
Jul 15 10:46:58.843357 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:46:58.843332    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod470d651d_0c05_4629_a564_50a940da19d1.slice/crio-0d2038907a9e974187b66da602225c3f06f7425a9c2cbc65c49141c355ae8924 WatchSource:0}: Error finding container 0d2038907a9e974187b66da602225c3f06f7425a9c2cbc65c49141c355ae8924: Status 404 returned error can't find the container with id 0d2038907a9e974187b66da602225c3f06f7425a9c2cbc65c49141c355ae8924
Jul 15 10:46:58.976964 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:58.976929    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-d68nc" event={"ID":"470d651d-0c05-4629-a564-50a940da19d1","Type":"ContainerStarted","Data":"0d2038907a9e974187b66da602225c3f06f7425a9c2cbc65c49141c355ae8924"}
Jul 15 10:46:58.978538 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:58.978514    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlj6f" event={"ID":"b144126b-086e-4cea-bc1a-3df5c25b8c35","Type":"ContainerStarted","Data":"1a80b08419d5f286ed9288c34bc6c9e226457b39ec130c2d24dabe7a3732a66f"}
Jul 15 10:46:58.979532 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:58.979514    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-znd4t" event={"ID":"bf9442c2-785b-449d-8cea-dee4b0564a08","Type":"ContainerStarted","Data":"495c1e453cb713ad8d40c7d33c1a5d9fd92049110a092a1f889fe435141240c0"}
Jul 15 10:46:58.980443 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:58.980424    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7" event={"ID":"b8a46b16-e479-4666-b59a-96902941c826","Type":"ContainerStarted","Data":"2d144ee7d87cc5a1fb1db52dc9dfe4d57e8dbae168f29b81eec2900c518a869e"}
Jul 15 10:46:59.030069 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.030034    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7768b4fff6-6rbwr"]
Jul 15 10:46:59.034028 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.034010    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.036705 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.036688    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 10:46:59.037248 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.037235    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f77v6\""
Jul 15 10:46:59.037355 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.037341    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 10:46:59.037583 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.037571    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 10:46:59.037693 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.037676    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 10:46:59.037816 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.037798    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 10:46:59.046577 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.046558    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7768b4fff6-6rbwr"]
Jul 15 10:46:59.149974 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.149944    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-config\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.149974 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.149968    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-service-ca\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.150131 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.150010    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk2hb\" (UniqueName: \"kubernetes.io/projected/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-kube-api-access-pk2hb\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.150131 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.150042    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-oauth-config\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.150131 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.150095    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-oauth-serving-cert\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.150131 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.150123    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-serving-cert\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.250774 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.250685    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk2hb\" (UniqueName: \"kubernetes.io/projected/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-kube-api-access-pk2hb\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.250774 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.250731    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-oauth-config\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.250774 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.250768    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-oauth-serving-cert\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.251012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.250794    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-serving-cert\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.251012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.250849    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-config\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.251012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.250870    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-service-ca\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.251707 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.251683    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-service-ca\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.251805 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.251685    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-oauth-serving-cert\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.251805 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.251685    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-config\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.253212 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.253190    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-oauth-config\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.253390 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.253373    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-serving-cert\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.259194 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.259171    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk2hb\" (UniqueName: \"kubernetes.io/projected/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-kube-api-access-pk2hb\") pod \"console-7768b4fff6-6rbwr\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") " pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.343595 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.343575    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:46:59.489712 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.489672    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7768b4fff6-6rbwr"]
Jul 15 10:46:59.493676 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:46:59.493644    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0398c8b6_ae37_459a_aa5c_7f406c4ee98c.slice/crio-19505c3a2e1c23741de0fe1a1cf0b448ace29ba9c5ef36cfa44fbbd967ce47db WatchSource:0}: Error finding container 19505c3a2e1c23741de0fe1a1cf0b448ace29ba9c5ef36cfa44fbbd967ce47db: Status 404 returned error can't find the container with id 19505c3a2e1c23741de0fe1a1cf0b448ace29ba9c5ef36cfa44fbbd967ce47db
Jul 15 10:46:59.987351 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:46:59.987312    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7768b4fff6-6rbwr" event={"ID":"0398c8b6-ae37-459a-aa5c-7f406c4ee98c","Type":"ContainerStarted","Data":"19505c3a2e1c23741de0fe1a1cf0b448ace29ba9c5ef36cfa44fbbd967ce47db"}
Jul 15 10:47:00.864830 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:00.864719    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:47:00.877454 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:00.877423    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") pod \"image-registry-7d85c67bb6-hlzhc\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") " pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:47:00.965716 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:00.965675    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:47:00.975002 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:00.974975    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b2f4d3f-9e4f-470d-88ab-59c274848f26-metrics-tls\") pod \"dns-default-52trx\" (UID: \"3b2f4d3f-9e4f-470d-88ab-59c274848f26\") " pod="openshift-dns/dns-default-52trx"
Jul 15 10:47:00.995391 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:00.994027    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-d68nc" event={"ID":"470d651d-0c05-4629-a564-50a940da19d1","Type":"ContainerStarted","Data":"0b8dc1d6afa1f5e2c2259338e8610e1322cde9475480510dd5d5bc9ce48ec02c"}
Jul 15 10:47:00.995391 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:00.995152    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mqg6r\""
Jul 15 10:47:01.002542 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:01.002293    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:47:01.012710 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:01.012597    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-d68nc" podStartSLOduration=2.706635329 podStartE2EDuration="4.012578693s" totalImagesPullingTime="1.305943364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:01.011769642 +0000 UTC m=+98.811243169" watchObservedRunningTime="2026-07-15 10:47:01.012578693 +0000 UTC m=+98.812052220"
Jul 15 10:47:01.015878 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:01.015683    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8cmqw\""
Jul 15 10:47:01.024164 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:01.023736    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-52trx"
Jul 15 10:47:01.219061 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:01.219024    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d85c67bb6-hlzhc"]
Jul 15 10:47:01.223006 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:47:01.222836    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf252f770_c18d_422d_b550_4157655338ba.slice/crio-3c2998d128803a89a459f049640cf6a01caeaa2c741830f53b48042bc792e6ed WatchSource:0}: Error finding container 3c2998d128803a89a459f049640cf6a01caeaa2c741830f53b48042bc792e6ed: Status 404 returned error can't find the container with id 3c2998d128803a89a459f049640cf6a01caeaa2c741830f53b48042bc792e6ed
Jul 15 10:47:01.226956 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:01.225115    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-52trx"]
Jul 15 10:47:01.233263 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:47:01.233235    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b2f4d3f_9e4f_470d_88ab_59c274848f26.slice/crio-168adc0dedf4a51b26ca1803072f0d3e34420924047e313f98b28d4fa72a86f2 WatchSource:0}: Error finding container 168adc0dedf4a51b26ca1803072f0d3e34420924047e313f98b28d4fa72a86f2: Status 404 returned error can't find the container with id 168adc0dedf4a51b26ca1803072f0d3e34420924047e313f98b28d4fa72a86f2
Jul 15 10:47:02.000751 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:02.000400    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc" event={"ID":"f252f770-c18d-422d-b550-4157655338ba","Type":"ContainerStarted","Data":"2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69"}
Jul 15 10:47:02.002257 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:02.002154    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:47:02.002257 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:02.002182    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc" event={"ID":"f252f770-c18d-422d-b550-4157655338ba","Type":"ContainerStarted","Data":"3c2998d128803a89a459f049640cf6a01caeaa2c741830f53b48042bc792e6ed"}
Jul 15 10:47:02.003551 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:02.003491    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-52trx" event={"ID":"3b2f4d3f-9e4f-470d-88ab-59c274848f26","Type":"ContainerStarted","Data":"168adc0dedf4a51b26ca1803072f0d3e34420924047e313f98b28d4fa72a86f2"}
Jul 15 10:47:02.025963 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:02.025922    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc" podStartSLOduration=78.025909742 podStartE2EDuration="1m18.025909742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:45:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:02.024745356 +0000 UTC m=+99.824218882" watchObservedRunningTime="2026-07-15 10:47:02.025909742 +0000 UTC m=+99.825383268"
Jul 15 10:47:04.899858 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:04.899322    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n9fpd"
Jul 15 10:47:07.606872 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.606838    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f47cfc596-cpcw5"]
Jul 15 10:47:07.609387 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.609360    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.617686 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.617536    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 10:47:07.625139 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.625115    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f47cfc596-cpcw5"]
Jul 15 10:47:07.725650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.725614    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-oauth-serving-cert\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.725814 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.725657    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-trusted-ca-bundle\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.725814 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.725694    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-oauth-config\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.725814 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.725745    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-service-ca\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.725814 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.725775    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-config\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.726016 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.725830    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76tv7\" (UniqueName: \"kubernetes.io/projected/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-kube-api-access-76tv7\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.726016 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.725937    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-serving-cert\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.827341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.826947    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-service-ca\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.827341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.826993    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-config\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.827341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.827040    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76tv7\" (UniqueName: \"kubernetes.io/projected/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-kube-api-access-76tv7\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.827341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.827089    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-serving-cert\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.827341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.827119    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-oauth-serving-cert\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.827341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.827144    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-trusted-ca-bundle\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.827341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.827183    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-oauth-config\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.827776 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.827757    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-service-ca\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.827832 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.827780    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-config\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.829366 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.829246    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-oauth-serving-cert\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.830102 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.830061    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-trusted-ca-bundle\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.831353 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.831333    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-serving-cert\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.831512 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.831494    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-oauth-config\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.838561 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.838543    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76tv7\" (UniqueName: \"kubernetes.io/projected/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-kube-api-access-76tv7\") pod \"console-5f47cfc596-cpcw5\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") " pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:07.922203 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:07.922012    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:08.030262 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:08.030222    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7" event={"ID":"b8a46b16-e479-4666-b59a-96902941c826","Type":"ContainerStarted","Data":"645b3f5c9c5ac9fc7085e8b1fbfd0d7a830e53dea60eac7becaba4664595868e"}
Jul 15 10:47:08.037579 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:08.036814    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-52trx" event={"ID":"3b2f4d3f-9e4f-470d-88ab-59c274848f26","Type":"ContainerStarted","Data":"b357e0ce098c2b9ce05190d7c80ea39213d2ed934b525ef1275777f4dbd29fbc"}
Jul 15 10:47:08.037579 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:08.036911    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-52trx" event={"ID":"3b2f4d3f-9e4f-470d-88ab-59c274848f26","Type":"ContainerStarted","Data":"50f507e874d3933a4287e380bc810351091330df64021f02ee443c928eea9d22"}
Jul 15 10:47:08.037579 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:08.037396    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-52trx"
Jul 15 10:47:08.040420 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:08.040395    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7768b4fff6-6rbwr" event={"ID":"0398c8b6-ae37-459a-aa5c-7f406c4ee98c","Type":"ContainerStarted","Data":"9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af"}
Jul 15 10:47:08.043825 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:08.043796    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vlj6f" event={"ID":"b144126b-086e-4cea-bc1a-3df5c25b8c35","Type":"ContainerStarted","Data":"983d0275aebfed7fb924b01d3424646a91fb19a15f17acd8aead119b66393dd9"}
Jul 15 10:47:08.053132 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:08.053016    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74878b79d6-7h8p7" podStartSLOduration=2.141014237 podStartE2EDuration="11.05299787s" totalImagesPullingTime="8.911983633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:08.051729646 +0000 UTC m=+105.851203172" watchObservedRunningTime="2026-07-15 10:47:08.05299787 +0000 UTC m=+105.852471396"
Jul 15 10:47:08.070859 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:08.070799    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-52trx" podStartSLOduration=64.538496851 podStartE2EDuration="1m11.070780272s" totalImagesPullingTime="6.532283421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:45:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:08.069864275 +0000 UTC m=+105.869337801" watchObservedRunningTime="2026-07-15 10:47:08.070780272 +0000 UTC m=+105.870253798"
Jul 15 10:47:08.090133 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:08.090089    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7768b4fff6-6rbwr" podStartSLOduration=0.856275634 podStartE2EDuration="9.090073213s" totalImagesPullingTime="8.233797579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:08.088602582 +0000 UTC m=+105.888076107" watchObservedRunningTime="2026-07-15 10:47:08.090073213 +0000 UTC m=+105.889546744"
Jul 15 10:47:08.093937 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:08.093912    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f47cfc596-cpcw5"]
Jul 15 10:47:08.097426 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:47:08.097399    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee18f1f_f19a_4d9c_bb3a_7bc5911463ac.slice/crio-e643340109dddc50f9bd4f9d823c57f55fe2a252e9c9aa4d2f6c9e71f285080c WatchSource:0}: Error finding container e643340109dddc50f9bd4f9d823c57f55fe2a252e9c9aa4d2f6c9e71f285080c: Status 404 returned error can't find the container with id e643340109dddc50f9bd4f9d823c57f55fe2a252e9c9aa4d2f6c9e71f285080c
Jul 15 10:47:08.114183 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:08.114144    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vlj6f" podStartSLOduration=16.682645291 podStartE2EDuration="34.114129056s" totalImagesPullingTime="17.431483765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 10:46:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:08.113524478 +0000 UTC m=+105.912998000" watchObservedRunningTime="2026-07-15 10:47:08.114129056 +0000 UTC m=+105.913602581"
Jul 15 10:47:09.049886 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:09.049767    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f47cfc596-cpcw5" event={"ID":"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac","Type":"ContainerStarted","Data":"7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8"}
Jul 15 10:47:09.049886 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:09.049833    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f47cfc596-cpcw5" event={"ID":"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac","Type":"ContainerStarted","Data":"e643340109dddc50f9bd4f9d823c57f55fe2a252e9c9aa4d2f6c9e71f285080c"}
Jul 15 10:47:09.071047 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:09.071001    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f47cfc596-cpcw5" podStartSLOduration=2.070987046 podStartE2EDuration="2.070987046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:47:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:09.069635605 +0000 UTC m=+106.869109130" watchObservedRunningTime="2026-07-15 10:47:09.070987046 +0000 UTC m=+106.870460570"
Jul 15 10:47:09.344430 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:09.344345    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:47:09.344430 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:09.344428    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:47:09.349911 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:09.349886    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:47:10.059978 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.059943    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:47:10.495984 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.495952    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"]
Jul 15 10:47:10.500449 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.500422    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.503879 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.503637    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dbcfw\""
Jul 15 10:47:10.503879 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.503637    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 10:47:10.504139 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.504121    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 15 10:47:10.504539 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.504509    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 15 10:47:10.510524 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.510499    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"]
Jul 15 10:47:10.551230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.551197    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f89c65b-cada-4f06-b034-72deee0905f8-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.551399 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.551240    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f89c65b-cada-4f06-b034-72deee0905f8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.551399 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.551286    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f89c65b-cada-4f06-b034-72deee0905f8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.551399 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.551366    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl8nw\" (UniqueName: \"kubernetes.io/projected/8f89c65b-cada-4f06-b034-72deee0905f8-kube-api-access-pl8nw\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.556235 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.556212    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zv49m"]
Jul 15 10:47:10.559702 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.559680    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.562024 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.561997    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 10:47:10.562156 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.561998    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 10:47:10.562507 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.562483    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 10:47:10.562845 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.562828    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q7hjh\""
Jul 15 10:47:10.651936 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.651903    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-wtmp\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.652096 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.651948    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl8nw\" (UniqueName: \"kubernetes.io/projected/8f89c65b-cada-4f06-b034-72deee0905f8-kube-api-access-pl8nw\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.652096 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.651972    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/363ab220-4030-4496-b9ed-9e86a7de501c-sys\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.652096 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.652009    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.652096 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.652057    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/363ab220-4030-4496-b9ed-9e86a7de501c-metrics-client-ca\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.652096 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.652076    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jldrc\" (UniqueName: \"kubernetes.io/projected/363ab220-4030-4496-b9ed-9e86a7de501c-kube-api-access-jldrc\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.652393 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.652186    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f89c65b-cada-4f06-b034-72deee0905f8-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.652393 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.652226    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-accelerators-collector-config\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.652393 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.652266    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-textfile\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.652393 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.652320    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f89c65b-cada-4f06-b034-72deee0905f8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.652393 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.652344    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/363ab220-4030-4496-b9ed-9e86a7de501c-root\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.652393 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.652374    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f89c65b-cada-4f06-b034-72deee0905f8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.652680 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.652403    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-tls\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.652966 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.652937    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f89c65b-cada-4f06-b034-72deee0905f8-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.655100 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.655059    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f89c65b-cada-4f06-b034-72deee0905f8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.655231 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.655209    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f89c65b-cada-4f06-b034-72deee0905f8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.671664 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.671638    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl8nw\" (UniqueName: \"kubernetes.io/projected/8f89c65b-cada-4f06-b034-72deee0905f8-kube-api-access-pl8nw\") pod \"openshift-state-metrics-84bd68b896-m5npz\" (UID: \"8f89c65b-cada-4f06-b034-72deee0905f8\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.753417 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.753267    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/363ab220-4030-4496-b9ed-9e86a7de501c-metrics-client-ca\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.753417 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.753338    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jldrc\" (UniqueName: \"kubernetes.io/projected/363ab220-4030-4496-b9ed-9e86a7de501c-kube-api-access-jldrc\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.753611 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.753424    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-accelerators-collector-config\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.753611 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.753457    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-textfile\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.753611 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.753487    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/363ab220-4030-4496-b9ed-9e86a7de501c-root\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.753611 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.753520    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-tls\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.753611 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.753541    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-wtmp\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.753611 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.753574    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/363ab220-4030-4496-b9ed-9e86a7de501c-sys\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.753611 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.753592    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.753904 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.753878    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/363ab220-4030-4496-b9ed-9e86a7de501c-metrics-client-ca\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.753976 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.753942    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-accelerators-collector-config\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.754038 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.754020    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/363ab220-4030-4496-b9ed-9e86a7de501c-sys\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.754093 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.754032    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-wtmp\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.754093 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.754067    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/363ab220-4030-4496-b9ed-9e86a7de501c-root\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.754193 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.754140    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-textfile\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.756212 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.756187    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-tls\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.756356 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.756218    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/363ab220-4030-4496-b9ed-9e86a7de501c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.764763 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.764732    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jldrc\" (UniqueName: \"kubernetes.io/projected/363ab220-4030-4496-b9ed-9e86a7de501c-kube-api-access-jldrc\") pod \"node-exporter-zv49m\" (UID: \"363ab220-4030-4496-b9ed-9e86a7de501c\") " pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:10.812551 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.812512    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"
Jul 15 10:47:10.870454 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:10.870412    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zv49m"
Jul 15 10:47:11.668896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.668863    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 10:47:11.672741 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.672717    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.676445 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.676424    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 15 10:47:11.676587 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.676562    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 15 10:47:11.676898 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.676882    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 15 10:47:11.677453 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.677436    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 15 10:47:11.678594 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.678573    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 15 10:47:11.678798 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.678783    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 15 10:47:11.678884 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.678821    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 15 10:47:11.679460 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.679440    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 15 10:47:11.679565 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.679504    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t5pxl\""
Jul 15 10:47:11.679565 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.679514    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 15 10:47:11.691196 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.691175    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 10:47:11.763413 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763374    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.763579 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763420    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-config-volume\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.763579 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763446    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.763579 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763486    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-config-out\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.763579 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763511    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.763579 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763537    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb8wf\" (UniqueName: \"kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-kube-api-access-qb8wf\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.763579 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763567    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.763882 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763618    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.763882 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763715    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-web-config\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.763882 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763788    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.763882 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763843    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.763882 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763869    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.764088 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.763917    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.864516 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864482    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.864516 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864520    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.864753 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864548    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.864753 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864585    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.864753 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864609    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-config-volume\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.864753 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864629    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.864753 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864665    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-config-out\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.864753 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864687    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.864753 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864715    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb8wf\" (UniqueName: \"kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-kube-api-access-qb8wf\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.864753 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864747    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.865128 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864777    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.865128 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864842    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-web-config\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.865128 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.864885    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.865629 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.865547    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.865859 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.865817    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.866768 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:47:11.866104    2536 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 15 10:47:11.866768 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:47:11.866166    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-main-tls podName:6e3722cc-84fb-4482-933d-7083120c61af nodeName:}" failed. No retries permitted until 2026-07-15 10:47:12.366146249 +0000 UTC m=+110.165619764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "6e3722cc-84fb-4482-933d-7083120c61af") : secret "alertmanager-main-tls" not found
Jul 15 10:47:11.866768 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.866228    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.868320 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.868259    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-config-volume\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.868524 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.868497    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.868806 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.868776    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-config-out\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.869712 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.869689    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.870089 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.870064    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-web-config\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.870173 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.870121    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.870550 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.870520    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.870935 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.870907    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:11.877008 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:11.876984    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb8wf\" (UniqueName: \"kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-kube-api-access-qb8wf\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:12.369548 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:12.369503    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:12.372504 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:12.372450    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:12.584542 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:12.584510    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:47:15.269027 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:15.268980    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f47cfc596-cpcw5"]
Jul 15 10:47:17.188980 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.188938    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 10:47:17.369100 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.369069    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 10:47:17.369300 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.369220    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.371903 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.371877    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 10:47:17.372703 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.372675    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 10:47:17.372828 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.372749    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 10:47:17.372975 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.372951    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7m26jock92p8b\""
Jul 15 10:47:17.373101 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.373067    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 10:47:17.373101 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.373071    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 10:47:17.373101 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.373078    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 10:47:17.373342 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.373147    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 10:47:17.373342 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.373082    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 10:47:17.373507 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.373443    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 10:47:17.373507 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.373444    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 10:47:17.373507 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.373488    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 10:47:17.373727 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.373550    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 10:47:17.373727 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.373705    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 10:47:17.375128 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.375112    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-g2kjz\""
Jul 15 10:47:17.378050 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.378028    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 10:47:17.381069 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.381053    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 10:47:17.417386 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417362    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417465 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417390    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417465 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417411    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417465 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417434    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417563 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417497    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417563 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417532    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417563 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417552    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417651 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417570    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417651 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417618    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417710 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417657    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417710 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417697    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417806 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417716    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417806 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417758    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417806 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417775    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417935 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417808    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417935 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417840    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tmn8\" (UniqueName: \"kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-kube-api-access-6tmn8\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417935 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417860    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417935 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417879    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.417935 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417907    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-config\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.418098 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.417946    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.462233 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:47:17.462205    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod363ab220_4030_4496_b9ed_9e86a7de501c.slice/crio-5a3c7e44f57c38bc3ca96f0b7709de9d57b5a89d5fd331295da2592bb6d1bcce WatchSource:0}: Error finding container 5a3c7e44f57c38bc3ca96f0b7709de9d57b5a89d5fd331295da2592bb6d1bcce: Status 404 returned error can't find the container with id 5a3c7e44f57c38bc3ca96f0b7709de9d57b5a89d5fd331295da2592bb6d1bcce
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.518920    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.518969    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.518998    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.519030    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.519057    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.519112    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.519138    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.519169    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.519195    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.519234    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.520041    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.519269    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tmn8\" (UniqueName: \"kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-kube-api-access-6tmn8\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.520500    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.520532    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.520571    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-config\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.520608    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.522127 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.520645    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.523128 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.520675    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.523128 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.520709    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.523128 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.520755    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.523128 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.520786    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.526058 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.524955    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.526058 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.525609    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.526058 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.525748    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.526058 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.525757    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.527023 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.526666    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.527023 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.526776    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.530656 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.530333    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.531146 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.531122    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.531618 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.531594    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.533399 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.533354    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.534880 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.534837    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.535456 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.535403    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.535456 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.535415    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.535652 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.535607    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.536063 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.535855    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.536063 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.536026    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-config\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.536176 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.536058    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.536689 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.536648    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.536796 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.536776    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tmn8\" (UniqueName: \"kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-kube-api-access-6tmn8\") pod \"prometheus-k8s-0\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.612761 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.612731    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 10:47:17.615869 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:47:17.615836    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e3722cc_84fb_4482_933d_7083120c61af.slice/crio-9af2d95f630b4c4df58757500aeb5f08f5065baa31db786a965bee03da469e23 WatchSource:0}: Error finding container 9af2d95f630b4c4df58757500aeb5f08f5065baa31db786a965bee03da469e23: Status 404 returned error can't find the container with id 9af2d95f630b4c4df58757500aeb5f08f5065baa31db786a965bee03da469e23
Jul 15 10:47:17.640904 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.640876    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz"]
Jul 15 10:47:17.680294 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.680255    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:17.732830 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:47:17.732804    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f89c65b_cada_4f06_b034_72deee0905f8.slice/crio-7348df8b292461cb167b3a1b0eb7d9bfaf8236d5eb9ba00ce0ccd705ba2fce12 WatchSource:0}: Error finding container 7348df8b292461cb167b3a1b0eb7d9bfaf8236d5eb9ba00ce0ccd705ba2fce12: Status 404 returned error can't find the container with id 7348df8b292461cb167b3a1b0eb7d9bfaf8236d5eb9ba00ce0ccd705ba2fce12
Jul 15 10:47:17.915211 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.915185    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 10:47:17.918803 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:47:17.918780    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b7e369c_7769_4450_ab38_ea7716ac457b.slice/crio-754ae51e15ca161af88afad05d8492c6f1434e184beed244d7d8263937909077 WatchSource:0}: Error finding container 754ae51e15ca161af88afad05d8492c6f1434e184beed244d7d8263937909077: Status 404 returned error can't find the container with id 754ae51e15ca161af88afad05d8492c6f1434e184beed244d7d8263937909077
Jul 15 10:47:17.923145 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:17.923076    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:18.052043 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:18.051971    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-52trx"
Jul 15 10:47:18.085075 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:18.084969    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerStarted","Data":"9af2d95f630b4c4df58757500aeb5f08f5065baa31db786a965bee03da469e23"}
Jul 15 10:47:18.086858 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:18.086495    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-znd4t" event={"ID":"bf9442c2-785b-449d-8cea-dee4b0564a08","Type":"ContainerStarted","Data":"e1b89c9a503ab01ad9b4e306cb38776379a183a3126584db634b24565c8e2844"}
Jul 15 10:47:18.086858 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:18.086823    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-znd4t"
Jul 15 10:47:18.088525 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:18.088489    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zv49m" event={"ID":"363ab220-4030-4496-b9ed-9e86a7de501c","Type":"ContainerStarted","Data":"5a3c7e44f57c38bc3ca96f0b7709de9d57b5a89d5fd331295da2592bb6d1bcce"}
Jul 15 10:47:18.090881 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:18.090861    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz" event={"ID":"8f89c65b-cada-4f06-b034-72deee0905f8","Type":"ContainerStarted","Data":"ffd835ec913d81bff786228d23c3cff61aaf87f64dcd5730be224d1307fa4001"}
Jul 15 10:47:18.091031 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:18.091021    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz" event={"ID":"8f89c65b-cada-4f06-b034-72deee0905f8","Type":"ContainerStarted","Data":"b164daf1d1b84e86c18f45089ad2d7c46a46e671c8723c16ab0978e485504142"}
Jul 15 10:47:18.091115 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:18.091107    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz" event={"ID":"8f89c65b-cada-4f06-b034-72deee0905f8","Type":"ContainerStarted","Data":"7348df8b292461cb167b3a1b0eb7d9bfaf8236d5eb9ba00ce0ccd705ba2fce12"}
Jul 15 10:47:18.092409 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:18.092392    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerStarted","Data":"754ae51e15ca161af88afad05d8492c6f1434e184beed244d7d8263937909077"}
Jul 15 10:47:18.104810 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:18.104779    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-znd4t"
Jul 15 10:47:18.124012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:18.123949    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-znd4t" podStartSLOduration=2.143862839 podStartE2EDuration="21.123930434s" totalImagesPullingTime="18.980067595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:18.113241336 +0000 UTC m=+115.912714862" watchObservedRunningTime="2026-07-15 10:47:18.123930434 +0000 UTC m=+115.923403960"
Jul 15 10:47:19.688366 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:19.688324    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d85c67bb6-hlzhc"]
Jul 15 10:47:19.693880 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:19.693854    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:47:20.104698 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:20.104662    2536 generic.go:356] "Generic (PLEG): container finished" podID="363ab220-4030-4496-b9ed-9e86a7de501c" containerID="2fe54f72ee1b0e74d0d3603bc00266ac315d9701a5a9b47def43b09c88e36f86" exitCode=0
Jul 15 10:47:20.105757 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:20.105730    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zv49m" event={"ID":"363ab220-4030-4496-b9ed-9e86a7de501c","Type":"ContainerDied","Data":"2fe54f72ee1b0e74d0d3603bc00266ac315d9701a5a9b47def43b09c88e36f86"}
Jul 15 10:47:21.110942 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:21.110854    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zv49m" event={"ID":"363ab220-4030-4496-b9ed-9e86a7de501c","Type":"ContainerStarted","Data":"77d2b915ab1b4dbd63d7d6ad93ec88fa1db68f0958d24d18b0e501e3ac5328d7"}
Jul 15 10:47:21.110942 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:21.110933    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zv49m" event={"ID":"363ab220-4030-4496-b9ed-9e86a7de501c","Type":"ContainerStarted","Data":"e06ff01883d0824ec1ed3942d8c571ba8c9445122a7f1869ec2f094dcbb23558"}
Jul 15 10:47:21.116008 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:21.115975    2536 generic.go:356] "Generic (PLEG): container finished" podID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerID="f7feb8a8a6ac4a66e6962e900655d611ca50ba5dfba59b160e8a92c556701aad" exitCode=0
Jul 15 10:47:21.116147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:21.116098    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerDied","Data":"f7feb8a8a6ac4a66e6962e900655d611ca50ba5dfba59b160e8a92c556701aad"}
Jul 15 10:47:21.118230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:21.118206    2536 generic.go:356] "Generic (PLEG): container finished" podID="6e3722cc-84fb-4482-933d-7083120c61af" containerID="751acf8ba3fbd876f7cce5fd19641d02d322ccf3e3ecf10de578a4e228b794d2" exitCode=0
Jul 15 10:47:21.118353 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:21.118253    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerDied","Data":"751acf8ba3fbd876f7cce5fd19641d02d322ccf3e3ecf10de578a4e228b794d2"}
Jul 15 10:47:21.132539 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:21.132482    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zv49m" podStartSLOduration=8.724929239 podStartE2EDuration="11.132469004s" totalImagesPullingTime="2.305873809s" totalInitContainerRuntime="101.665956ms" isStatefulPod=true podCreationTimestamp="2026-07-15 10:47:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:21.131498942 +0000 UTC m=+118.930972468" watchObservedRunningTime="2026-07-15 10:47:21.132469004 +0000 UTC m=+118.931942528"
Jul 15 10:47:22.128123 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:22.128063    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz" event={"ID":"8f89c65b-cada-4f06-b034-72deee0905f8","Type":"ContainerStarted","Data":"a170007ab8bbbc1bb68c65290b4abe5acb449221d3dd3509ac6f2d60a5e7b0dd"}
Jul 15 10:47:26.152747 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:26.152401    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerStarted","Data":"a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02"}
Jul 15 10:47:26.152747 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:26.152496    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerStarted","Data":"551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0"}
Jul 15 10:47:26.152747 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:26.152534    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerStarted","Data":"5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c"}
Jul 15 10:47:26.152747 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:26.152568    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerStarted","Data":"fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb"}
Jul 15 10:47:26.152747 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:26.152601    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerStarted","Data":"4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c"}
Jul 15 10:47:26.155387 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:26.155358    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerStarted","Data":"9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda"}
Jul 15 10:47:26.155497 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:26.155423    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerStarted","Data":"08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040"}
Jul 15 10:47:26.272484 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:26.272431    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-m5npz" podStartSLOduration=12.838607139 podStartE2EDuration="16.272406993s" totalImagesPullingTime="3.433799854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:47:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:22.147961566 +0000 UTC m=+119.947435092" watchObservedRunningTime="2026-07-15 10:47:26.272406993 +0000 UTC m=+124.071880519"
Jul 15 10:47:26.273845 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:26.273803    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7768b4fff6-6rbwr"]
Jul 15 10:47:28.168225 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:28.168193    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerStarted","Data":"766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2"}
Jul 15 10:47:28.194766 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:28.194722    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=8.839349665 podStartE2EDuration="17.194708628s" totalImagesPullingTime="8.340105559s" totalInitContainerRuntime="15.253404ms" isStatefulPod=false podCreationTimestamp="2026-07-15 10:47:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:28.19370161 +0000 UTC m=+125.993175134" watchObservedRunningTime="2026-07-15 10:47:28.194708628 +0000 UTC m=+125.994182152"
Jul 15 10:47:29.175191 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:29.175152    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerStarted","Data":"d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501"}
Jul 15 10:47:29.175643 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:29.175234    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerStarted","Data":"60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897"}
Jul 15 10:47:29.175643 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:29.175266    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerStarted","Data":"d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564"}
Jul 15 10:47:29.175643 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:29.175324    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerStarted","Data":"b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5"}
Jul 15 10:47:29.214875 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:29.214818    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.240565355 podStartE2EDuration="12.214802463s" totalImagesPullingTime="8.965389256s" totalInitContainerRuntime="8.847852ms" isStatefulPod=false podCreationTimestamp="2026-07-15 10:47:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:29.212145907 +0000 UTC m=+127.011619445" watchObservedRunningTime="2026-07-15 10:47:29.214802463 +0000 UTC m=+127.014275988"
Jul 15 10:47:31.464601 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:31.464556    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:47:31.466985 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:31.466962    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f99bc36-1c09-41cc-9960-e8cf593e3e9c-metrics-certs\") pod \"network-metrics-daemon-6tndv\" (UID: \"4f99bc36-1c09-41cc-9960-e8cf593e3e9c\") " pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:47:31.607841 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:31.607807    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfsm8\""
Jul 15 10:47:31.615667 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:31.615637    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6tndv"
Jul 15 10:47:31.800143 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:31.800119    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6tndv"]
Jul 15 10:47:31.802739 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:47:31.802704    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f99bc36_1c09_41cc_9960_e8cf593e3e9c.slice/crio-24b218676a68ce58a24195452dd0e2bc5e57898bb12c9ee61d789c3c161d4927 WatchSource:0}: Error finding container 24b218676a68ce58a24195452dd0e2bc5e57898bb12c9ee61d789c3c161d4927: Status 404 returned error can't find the container with id 24b218676a68ce58a24195452dd0e2bc5e57898bb12c9ee61d789c3c161d4927
Jul 15 10:47:32.189597 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:32.189514    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6tndv" event={"ID":"4f99bc36-1c09-41cc-9960-e8cf593e3e9c","Type":"ContainerStarted","Data":"24b218676a68ce58a24195452dd0e2bc5e57898bb12c9ee61d789c3c161d4927"}
Jul 15 10:47:32.681402 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:32.681373    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:47:35.203493 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:35.203460    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6tndv" event={"ID":"4f99bc36-1c09-41cc-9960-e8cf593e3e9c","Type":"ContainerStarted","Data":"2ee226ed1ae07d3ad2829797e4e74bb0554d2052e25a449538e0b32f29f92e14"}
Jul 15 10:47:35.203834 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:35.203509    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6tndv" event={"ID":"4f99bc36-1c09-41cc-9960-e8cf593e3e9c","Type":"ContainerStarted","Data":"944a3aff00f99905c8785a4d08a7e4772bd7dc4c6ac032f05b86eb2ee60560d2"}
Jul 15 10:47:35.221067 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:35.221023    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6tndv" podStartSLOduration=130.794391642 podStartE2EDuration="2m13.221005461s" totalImagesPullingTime="2.426613819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:47:35.220150964 +0000 UTC m=+133.019624483" watchObservedRunningTime="2026-07-15 10:47:35.221005461 +0000 UTC m=+133.020478988"
Jul 15 10:47:40.295221 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.295164    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f47cfc596-cpcw5" podUID="cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" containerName="console" containerID="cri-o://7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8" gracePeriod=15
Jul 15 10:47:40.561523 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.561459    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:40.636704 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.636682    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76tv7\" (UniqueName: \"kubernetes.io/projected/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-kube-api-access-76tv7\") pod \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") "
Jul 15 10:47:40.636840 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.636725    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-trusted-ca-bundle\") pod \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") "
Jul 15 10:47:40.636840 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.636749    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-oauth-serving-cert\") pod \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") "
Jul 15 10:47:40.636840 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.636764    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-config\") pod \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") "
Jul 15 10:47:40.636840 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.636796    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-oauth-config\") pod \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") "
Jul 15 10:47:40.636840 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.636829    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-service-ca\") pod \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") "
Jul 15 10:47:40.637082 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.636874    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-serving-cert\") pod \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\" (UID: \"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac\") "
Jul 15 10:47:40.637189 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.637162    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" (UID: "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:47:40.637296 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.637164    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" (UID: "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:47:40.637296 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.637237    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-config" (OuterVolumeSpecName: "console-config") pod "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" (UID: "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:47:40.637296 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.637245    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-service-ca" (OuterVolumeSpecName: "service-ca") pod "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" (UID: "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:47:40.639125 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.639094    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-kube-api-access-76tv7" (OuterVolumeSpecName: "kube-api-access-76tv7") pod "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" (UID: "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac"). InnerVolumeSpecName "kube-api-access-76tv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 10:47:40.639217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.639099    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" (UID: "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:47:40.639217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.639158    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" (UID: "cee18f1f-f19a-4d9c-bb3a-7bc5911463ac"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:47:40.737593 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.737566    2536 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-oauth-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:40.737593 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.737591    2536 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-service-ca\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:40.737735 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.737600    2536 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:40.737735 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.737609    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76tv7\" (UniqueName: \"kubernetes.io/projected/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-kube-api-access-76tv7\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:40.737735 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.737617    2536 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-trusted-ca-bundle\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:40.737735 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.737625    2536 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-oauth-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:40.737735 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:40.737634    2536 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac-console-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:41.228623 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:41.228589    2536 generic.go:356] "Generic (PLEG): container finished" podID="cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" containerID="7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8" exitCode=0
Jul 15 10:47:41.228623 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:41.228622    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f47cfc596-cpcw5"
Jul 15 10:47:41.228860 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:41.228678    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f47cfc596-cpcw5" event={"ID":"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac","Type":"ContainerDied","Data":"7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8"}
Jul 15 10:47:41.228860 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:41.228733    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f47cfc596-cpcw5" event={"ID":"cee18f1f-f19a-4d9c-bb3a-7bc5911463ac","Type":"ContainerDied","Data":"e643340109dddc50f9bd4f9d823c57f55fe2a252e9c9aa4d2f6c9e71f285080c"}
Jul 15 10:47:41.228860 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:41.228754    2536 scope.go:117] "RemoveContainer" containerID="7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8"
Jul 15 10:47:41.237458 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:41.237440    2536 scope.go:117] "RemoveContainer" containerID="7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8"
Jul 15 10:47:41.237712 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:47:41.237693    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8\": container with ID starting with 7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8 not found: ID does not exist" containerID="7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8"
Jul 15 10:47:41.237758 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:41.237722    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8"} err="failed to get container status \"7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8\": rpc error: code = NotFound desc = could not find container \"7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8\": container with ID starting with 7e91249bdbd462452b5567542d5fad0c29f58e572d9a2defb5b282bfe368f5a8 not found: ID does not exist"
Jul 15 10:47:41.252397 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:41.252379    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f47cfc596-cpcw5"]
Jul 15 10:47:41.256155 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:41.256135    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f47cfc596-cpcw5"]
Jul 15 10:47:42.594058 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:42.594027    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" path="/var/lib/kubelet/pods/cee18f1f-f19a-4d9c-bb3a-7bc5911463ac/volumes"
Jul 15 10:47:44.713793 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:44.713740    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc" podUID="f252f770-c18d-422d-b550-4157655338ba" containerName="registry" containerID="cri-o://2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69" gracePeriod=30
Jul 15 10:47:45.009683 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.009657    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:47:45.073700 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.073673    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f252f770-c18d-422d-b550-4157655338ba-ca-trust-extracted\") pod \"f252f770-c18d-422d-b550-4157655338ba\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") "
Jul 15 10:47:45.073842 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.073706    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-image-registry-private-configuration\") pod \"f252f770-c18d-422d-b550-4157655338ba\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") "
Jul 15 10:47:45.073842 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.073737    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-trusted-ca\") pod \"f252f770-c18d-422d-b550-4157655338ba\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") "
Jul 15 10:47:45.073842 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.073772    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnsdv\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-kube-api-access-tnsdv\") pod \"f252f770-c18d-422d-b550-4157655338ba\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") "
Jul 15 10:47:45.073842 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.073821    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-bound-sa-token\") pod \"f252f770-c18d-422d-b550-4157655338ba\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") "
Jul 15 10:47:45.074057 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.073862    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") pod \"f252f770-c18d-422d-b550-4157655338ba\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") "
Jul 15 10:47:45.074057 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.073895    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-installation-pull-secrets\") pod \"f252f770-c18d-422d-b550-4157655338ba\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") "
Jul 15 10:47:45.074057 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.073919    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-registry-certificates\") pod \"f252f770-c18d-422d-b550-4157655338ba\" (UID: \"f252f770-c18d-422d-b550-4157655338ba\") "
Jul 15 10:47:45.074263 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.074231    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f252f770-c18d-422d-b550-4157655338ba" (UID: "f252f770-c18d-422d-b550-4157655338ba"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:47:45.074826 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.074804    2536 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-trusted-ca\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:45.075189 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.075042    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f252f770-c18d-422d-b550-4157655338ba" (UID: "f252f770-c18d-422d-b550-4157655338ba"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:47:45.076596 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.076567    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f252f770-c18d-422d-b550-4157655338ba" (UID: "f252f770-c18d-422d-b550-4157655338ba"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 10:47:45.076596 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.076584    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f252f770-c18d-422d-b550-4157655338ba" (UID: "f252f770-c18d-422d-b550-4157655338ba"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:47:45.076731 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.076630    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f252f770-c18d-422d-b550-4157655338ba" (UID: "f252f770-c18d-422d-b550-4157655338ba"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 10:47:45.076784 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.076739    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-kube-api-access-tnsdv" (OuterVolumeSpecName: "kube-api-access-tnsdv") pod "f252f770-c18d-422d-b550-4157655338ba" (UID: "f252f770-c18d-422d-b550-4157655338ba"). InnerVolumeSpecName "kube-api-access-tnsdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 10:47:45.076828 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.076776    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f252f770-c18d-422d-b550-4157655338ba" (UID: "f252f770-c18d-422d-b550-4157655338ba"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:47:45.083124 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.083102    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f252f770-c18d-422d-b550-4157655338ba-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f252f770-c18d-422d-b550-4157655338ba" (UID: "f252f770-c18d-422d-b550-4157655338ba"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 10:47:45.176261 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.176236    2536 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f252f770-c18d-422d-b550-4157655338ba-ca-trust-extracted\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:45.176261 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.176260    2536 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-image-registry-private-configuration\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:45.176410 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.176292    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnsdv\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-kube-api-access-tnsdv\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:45.176410 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.176302    2536 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-bound-sa-token\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:45.176410 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.176311    2536 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f252f770-c18d-422d-b550-4157655338ba-registry-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:45.176410 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.176318    2536 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f252f770-c18d-422d-b550-4157655338ba-installation-pull-secrets\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:45.176410 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.176326    2536 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f252f770-c18d-422d-b550-4157655338ba-registry-certificates\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:45.247391 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.247330    2536 generic.go:356] "Generic (PLEG): container finished" podID="f252f770-c18d-422d-b550-4157655338ba" containerID="2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69" exitCode=0
Jul 15 10:47:45.247391 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.247381    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc"
Jul 15 10:47:45.247506 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.247404    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc" event={"ID":"f252f770-c18d-422d-b550-4157655338ba","Type":"ContainerDied","Data":"2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69"}
Jul 15 10:47:45.247506 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.247452    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d85c67bb6-hlzhc" event={"ID":"f252f770-c18d-422d-b550-4157655338ba","Type":"ContainerDied","Data":"3c2998d128803a89a459f049640cf6a01caeaa2c741830f53b48042bc792e6ed"}
Jul 15 10:47:45.247506 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.247481    2536 scope.go:117] "RemoveContainer" containerID="2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69"
Jul 15 10:47:45.255846 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.255824    2536 scope.go:117] "RemoveContainer" containerID="2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69"
Jul 15 10:47:45.256087 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:47:45.256069    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69\": container with ID starting with 2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69 not found: ID does not exist" containerID="2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69"
Jul 15 10:47:45.256149 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.256099    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69"} err="failed to get container status \"2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69\": rpc error: code = NotFound desc = could not find container \"2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69\": container with ID starting with 2fa7d10cc546fe01db4cbfc5965c1fc7e20946c04f077da0687ea110f298fb69 not found: ID does not exist"
Jul 15 10:47:45.267924 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.267903    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d85c67bb6-hlzhc"]
Jul 15 10:47:45.272786 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:45.272764    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d85c67bb6-hlzhc"]
Jul 15 10:47:46.594912 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:46.594878    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f252f770-c18d-422d-b550-4157655338ba" path="/var/lib/kubelet/pods/f252f770-c18d-422d-b550-4157655338ba/volumes"
Jul 15 10:47:51.298384 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.298308    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7768b4fff6-6rbwr" podUID="0398c8b6-ae37-459a-aa5c-7f406c4ee98c" containerName="console" containerID="cri-o://9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af" gracePeriod=15
Jul 15 10:47:51.576774 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.576721    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:47:51.625725 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.625702    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-oauth-serving-cert\") pod \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") "
Jul 15 10:47:51.625882 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.625739    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-serving-cert\") pod \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") "
Jul 15 10:47:51.625882 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.625763    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-service-ca\") pod \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") "
Jul 15 10:47:51.625882 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.625808    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-oauth-config\") pod \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") "
Jul 15 10:47:51.625882 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.625830    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk2hb\" (UniqueName: \"kubernetes.io/projected/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-kube-api-access-pk2hb\") pod \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") "
Jul 15 10:47:51.625882 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.625855    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-config\") pod \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\" (UID: \"0398c8b6-ae37-459a-aa5c-7f406c4ee98c\") "
Jul 15 10:47:51.626443 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.626207    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0398c8b6-ae37-459a-aa5c-7f406c4ee98c" (UID: "0398c8b6-ae37-459a-aa5c-7f406c4ee98c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:47:51.626443 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.626256    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-service-ca" (OuterVolumeSpecName: "service-ca") pod "0398c8b6-ae37-459a-aa5c-7f406c4ee98c" (UID: "0398c8b6-ae37-459a-aa5c-7f406c4ee98c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:47:51.626443 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.626336    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-config" (OuterVolumeSpecName: "console-config") pod "0398c8b6-ae37-459a-aa5c-7f406c4ee98c" (UID: "0398c8b6-ae37-459a-aa5c-7f406c4ee98c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:47:51.628036 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.628013    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-kube-api-access-pk2hb" (OuterVolumeSpecName: "kube-api-access-pk2hb") pod "0398c8b6-ae37-459a-aa5c-7f406c4ee98c" (UID: "0398c8b6-ae37-459a-aa5c-7f406c4ee98c"). InnerVolumeSpecName "kube-api-access-pk2hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 10:47:51.628517 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.628491    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0398c8b6-ae37-459a-aa5c-7f406c4ee98c" (UID: "0398c8b6-ae37-459a-aa5c-7f406c4ee98c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:47:51.628517 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.628503    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0398c8b6-ae37-459a-aa5c-7f406c4ee98c" (UID: "0398c8b6-ae37-459a-aa5c-7f406c4ee98c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:47:51.727368 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.727344    2536 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-oauth-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:51.727368 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.727366    2536 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-serving-cert\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:51.727491 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.727376    2536 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-service-ca\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:51.727491 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.727384    2536 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-oauth-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:51.727491 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.727392    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk2hb\" (UniqueName: \"kubernetes.io/projected/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-kube-api-access-pk2hb\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:51.727491 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:51.727401    2536 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0398c8b6-ae37-459a-aa5c-7f406c4ee98c-console-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:47:52.276324 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:52.276294    2536 generic.go:356] "Generic (PLEG): container finished" podID="0398c8b6-ae37-459a-aa5c-7f406c4ee98c" containerID="9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af" exitCode=0
Jul 15 10:47:52.276471 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:52.276339    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7768b4fff6-6rbwr"
Jul 15 10:47:52.276471 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:52.276361    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7768b4fff6-6rbwr" event={"ID":"0398c8b6-ae37-459a-aa5c-7f406c4ee98c","Type":"ContainerDied","Data":"9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af"}
Jul 15 10:47:52.276471 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:52.276402    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7768b4fff6-6rbwr" event={"ID":"0398c8b6-ae37-459a-aa5c-7f406c4ee98c","Type":"ContainerDied","Data":"19505c3a2e1c23741de0fe1a1cf0b448ace29ba9c5ef36cfa44fbbd967ce47db"}
Jul 15 10:47:52.276471 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:52.276420    2536 scope.go:117] "RemoveContainer" containerID="9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af"
Jul 15 10:47:52.284691 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:52.284671    2536 scope.go:117] "RemoveContainer" containerID="9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af"
Jul 15 10:47:52.284931 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:47:52.284913    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af\": container with ID starting with 9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af not found: ID does not exist" containerID="9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af"
Jul 15 10:47:52.284990 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:52.284946    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af"} err="failed to get container status \"9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af\": rpc error: code = NotFound desc = could not find container \"9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af\": container with ID starting with 9bff7392221fa4546590aaa2a06e700d14228d0643442d480d4ab71d8df900af not found: ID does not exist"
Jul 15 10:47:52.296634 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:52.296613    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7768b4fff6-6rbwr"]
Jul 15 10:47:52.300293 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:52.300258    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7768b4fff6-6rbwr"]
Jul 15 10:47:52.596086 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:47:52.596008    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0398c8b6-ae37-459a-aa5c-7f406c4ee98c" path="/var/lib/kubelet/pods/0398c8b6-ae37-459a-aa5c-7f406c4ee98c/volumes"
Jul 15 10:48:04.321579 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:04.321548    2536 generic.go:356] "Generic (PLEG): container finished" podID="c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9" containerID="116e90edb65d3e056f669b85222095bee93f93cd33b2591a8dfb315895a592aa" exitCode=0
Jul 15 10:48:04.321956 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:04.321599    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw" event={"ID":"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9","Type":"ContainerDied","Data":"116e90edb65d3e056f669b85222095bee93f93cd33b2591a8dfb315895a592aa"}
Jul 15 10:48:04.321956 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:04.321900    2536 scope.go:117] "RemoveContainer" containerID="116e90edb65d3e056f669b85222095bee93f93cd33b2591a8dfb315895a592aa"
Jul 15 10:48:05.326908 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:05.326875    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-tcwtw" event={"ID":"c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9","Type":"ContainerStarted","Data":"0c4eac087c828914be0d408f85c251d8a8a629621dbd76b7b464c6de62692b30"}
Jul 15 10:48:17.681536 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:17.681433    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:17.707948 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:17.707924    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:18.395983 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:18.395704    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:31.255409 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.255377    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 10:48:31.255930 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.255717    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="alertmanager" containerID="cri-o://4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c" gracePeriod=120
Jul 15 10:48:31.255930 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.255749    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy-metric" containerID="cri-o://a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02" gracePeriod=120
Jul 15 10:48:31.255930 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.255780    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy-web" containerID="cri-o://5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c" gracePeriod=120
Jul 15 10:48:31.255930 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.255803    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="config-reloader" containerID="cri-o://fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb" gracePeriod=120
Jul 15 10:48:31.255930 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.255817    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="prom-label-proxy" containerID="cri-o://766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2" gracePeriod=120
Jul 15 10:48:31.256585 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.256496    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy" containerID="cri-o://551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0" gracePeriod=120
Jul 15 10:48:31.429746 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.429714    2536 generic.go:356] "Generic (PLEG): container finished" podID="6e3722cc-84fb-4482-933d-7083120c61af" containerID="766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2" exitCode=0
Jul 15 10:48:31.429746 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.429741    2536 generic.go:356] "Generic (PLEG): container finished" podID="6e3722cc-84fb-4482-933d-7083120c61af" containerID="551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0" exitCode=0
Jul 15 10:48:31.429746 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.429747    2536 generic.go:356] "Generic (PLEG): container finished" podID="6e3722cc-84fb-4482-933d-7083120c61af" containerID="fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb" exitCode=0
Jul 15 10:48:31.429907 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.429753    2536 generic.go:356] "Generic (PLEG): container finished" podID="6e3722cc-84fb-4482-933d-7083120c61af" containerID="4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c" exitCode=0
Jul 15 10:48:31.429907 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.429787    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerDied","Data":"766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2"}
Jul 15 10:48:31.429907 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.429872    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerDied","Data":"551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0"}
Jul 15 10:48:31.429907 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.429899    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerDied","Data":"fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb"}
Jul 15 10:48:31.430025 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:31.429922    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerDied","Data":"4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c"}
Jul 15 10:48:32.436440 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.436407    2536 generic.go:356] "Generic (PLEG): container finished" podID="6e3722cc-84fb-4482-933d-7083120c61af" containerID="5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c" exitCode=0
Jul 15 10:48:32.436727 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.436470    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerDied","Data":"5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c"}
Jul 15 10:48:32.613591 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.613570    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:32.733929 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.733901    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-config-out\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734072 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.733942    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-metrics-client-ca\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734072 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.733972    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-web\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734072 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734001    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-main-tls\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734072 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734041    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-cluster-tls-config\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734072    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-web-config\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734109    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qb8wf\" (UniqueName: \"kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-kube-api-access-qb8wf\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734154    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-main-db\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734184    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734206    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-trusted-ca-bundle\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734236    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-tls-assets\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734259    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-config-volume\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734814 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734312    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-metric\") pod \"6e3722cc-84fb-4482-933d-7083120c61af\" (UID: \"6e3722cc-84fb-4482-933d-7083120c61af\") "
Jul 15 10:48:32.734814 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734474    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 10:48:32.734814 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734609    2536 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-main-db\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.735150 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.734912    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:48:32.735150 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.735112    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:48:32.737186 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.737144    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:32.737645 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.737611    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-kube-api-access-qb8wf" (OuterVolumeSpecName: "kube-api-access-qb8wf") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "kube-api-access-qb8wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 10:48:32.737759 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.737722    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-config-out" (OuterVolumeSpecName: "config-out") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 10:48:32.737759 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.737740    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:32.738262 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.738223    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:32.738460 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.738438    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-config-volume" (OuterVolumeSpecName: "config-volume") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:32.738611 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.738590    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 10:48:32.738847 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.738830    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:32.742018 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.741991    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:32.747775 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.747752    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-web-config" (OuterVolumeSpecName: "web-config") pod "6e3722cc-84fb-4482-933d-7083120c61af" (UID: "6e3722cc-84fb-4482-933d-7083120c61af"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:32.835648 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835615    2536 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-cluster-tls-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.835648 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835647    2536 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-web-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.835648 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835658    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qb8wf\" (UniqueName: \"kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-kube-api-access-qb8wf\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.835804 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835667    2536 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.835804 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835677    2536 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.835804 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835685    2536 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e3722cc-84fb-4482-933d-7083120c61af-tls-assets\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.835804 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835692    2536 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-config-volume\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.835804 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835700    2536 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.835804 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835708    2536 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e3722cc-84fb-4482-933d-7083120c61af-config-out\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.835804 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835716    2536 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e3722cc-84fb-4482-933d-7083120c61af-metrics-client-ca\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.835804 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835725    2536 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:32.835804 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:32.835734    2536 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6e3722cc-84fb-4482-933d-7083120c61af-secret-alertmanager-main-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:33.443716 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.443689    2536 generic.go:356] "Generic (PLEG): container finished" podID="6e3722cc-84fb-4482-933d-7083120c61af" containerID="a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02" exitCode=0
Jul 15 10:48:33.444096 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.443737    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerDied","Data":"a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02"}
Jul 15 10:48:33.444096 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.443758    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.444096 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.443816    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6e3722cc-84fb-4482-933d-7083120c61af","Type":"ContainerDied","Data":"9af2d95f630b4c4df58757500aeb5f08f5065baa31db786a965bee03da469e23"}
Jul 15 10:48:33.444096 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.443868    2536 scope.go:117] "RemoveContainer" containerID="766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2"
Jul 15 10:48:33.450959 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.450942    2536 scope.go:117] "RemoveContainer" containerID="a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02"
Jul 15 10:48:33.460744 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.457998    2536 scope.go:117] "RemoveContainer" containerID="551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0"
Jul 15 10:48:33.468186 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.468170    2536 scope.go:117] "RemoveContainer" containerID="5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c"
Jul 15 10:48:33.473352 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.473322    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 10:48:33.475376 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.475356    2536 scope.go:117] "RemoveContainer" containerID="fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb"
Jul 15 10:48:33.478486 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.478465    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 10:48:33.482463 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.482447    2536 scope.go:117] "RemoveContainer" containerID="4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c"
Jul 15 10:48:33.489166 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.489150    2536 scope.go:117] "RemoveContainer" containerID="751acf8ba3fbd876f7cce5fd19641d02d322ccf3e3ecf10de578a4e228b794d2"
Jul 15 10:48:33.495536 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.495522    2536 scope.go:117] "RemoveContainer" containerID="766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2"
Jul 15 10:48:33.495773 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:33.495757    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2\": container with ID starting with 766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2 not found: ID does not exist" containerID="766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2"
Jul 15 10:48:33.495809 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.495780    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2"} err="failed to get container status \"766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2\": rpc error: code = NotFound desc = could not find container \"766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2\": container with ID starting with 766b373bc3ba586f56edabd5f95c9df0579a5d52b3449bd401ef964a8abfe6f2 not found: ID does not exist"
Jul 15 10:48:33.495809 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.495803    2536 scope.go:117] "RemoveContainer" containerID="a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02"
Jul 15 10:48:33.496027 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:33.496010    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02\": container with ID starting with a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02 not found: ID does not exist" containerID="a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02"
Jul 15 10:48:33.496066 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.496034    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02"} err="failed to get container status \"a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02\": rpc error: code = NotFound desc = could not find container \"a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02\": container with ID starting with a8e8957cc92754874ceea53aa904c2e8f57418bde7ba19eb554571fe064aba02 not found: ID does not exist"
Jul 15 10:48:33.496066 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.496050    2536 scope.go:117] "RemoveContainer" containerID="551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0"
Jul 15 10:48:33.496245 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:33.496232    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0\": container with ID starting with 551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0 not found: ID does not exist" containerID="551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0"
Jul 15 10:48:33.496356 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.496247    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0"} err="failed to get container status \"551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0\": rpc error: code = NotFound desc = could not find container \"551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0\": container with ID starting with 551a5d79becf467a7fff3e7eea78dc0465066e8e5d70bd6a31ced962f56039f0 not found: ID does not exist"
Jul 15 10:48:33.496356 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.496258    2536 scope.go:117] "RemoveContainer" containerID="5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c"
Jul 15 10:48:33.496476 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:33.496460    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c\": container with ID starting with 5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c not found: ID does not exist" containerID="5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c"
Jul 15 10:48:33.496560 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.496481    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c"} err="failed to get container status \"5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c\": rpc error: code = NotFound desc = could not find container \"5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c\": container with ID starting with 5148ed04791d2eec2f841e41084acc27ee277eead62fb4d3b8362dda09f7dc2c not found: ID does not exist"
Jul 15 10:48:33.496560 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.496494    2536 scope.go:117] "RemoveContainer" containerID="fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb"
Jul 15 10:48:33.496691 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:33.496676    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb\": container with ID starting with fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb not found: ID does not exist" containerID="fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb"
Jul 15 10:48:33.496728 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.496695    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb"} err="failed to get container status \"fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb\": rpc error: code = NotFound desc = could not find container \"fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb\": container with ID starting with fce53b95f76582ba4091259d216aee27a4a5b2d7640d457acfad37b2ea1addeb not found: ID does not exist"
Jul 15 10:48:33.496728 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.496708    2536 scope.go:117] "RemoveContainer" containerID="4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c"
Jul 15 10:48:33.496885 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:33.496869    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c\": container with ID starting with 4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c not found: ID does not exist" containerID="4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c"
Jul 15 10:48:33.496924 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.496888    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c"} err="failed to get container status \"4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c\": rpc error: code = NotFound desc = could not find container \"4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c\": container with ID starting with 4761adbebe4ec0409c6fd84fc0089abb53a51af5ba10ba7a259ea2dfda2c765c not found: ID does not exist"
Jul 15 10:48:33.496924 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.496901    2536 scope.go:117] "RemoveContainer" containerID="751acf8ba3fbd876f7cce5fd19641d02d322ccf3e3ecf10de578a4e228b794d2"
Jul 15 10:48:33.497113 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:33.497095    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"751acf8ba3fbd876f7cce5fd19641d02d322ccf3e3ecf10de578a4e228b794d2\": container with ID starting with 751acf8ba3fbd876f7cce5fd19641d02d322ccf3e3ecf10de578a4e228b794d2 not found: ID does not exist" containerID="751acf8ba3fbd876f7cce5fd19641d02d322ccf3e3ecf10de578a4e228b794d2"
Jul 15 10:48:33.497160 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.497115    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"751acf8ba3fbd876f7cce5fd19641d02d322ccf3e3ecf10de578a4e228b794d2"} err="failed to get container status \"751acf8ba3fbd876f7cce5fd19641d02d322ccf3e3ecf10de578a4e228b794d2\": rpc error: code = NotFound desc = could not find container \"751acf8ba3fbd876f7cce5fd19641d02d322ccf3e3ecf10de578a4e228b794d2\": container with ID starting with 751acf8ba3fbd876f7cce5fd19641d02d322ccf3e3ecf10de578a4e228b794d2 not found: ID does not exist"
Jul 15 10:48:33.507452 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507428    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 10:48:33.507717 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507704    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="prom-label-proxy"
Jul 15 10:48:33.507755 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507719    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="prom-label-proxy"
Jul 15 10:48:33.507755 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507731    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy"
Jul 15 10:48:33.507755 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507736    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy"
Jul 15 10:48:33.507755 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507745    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0398c8b6-ae37-459a-aa5c-7f406c4ee98c" containerName="console"
Jul 15 10:48:33.507755 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507750    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="0398c8b6-ae37-459a-aa5c-7f406c4ee98c" containerName="console"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507757    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" containerName="console"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507762    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" containerName="console"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507768    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="config-reloader"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507773    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="config-reloader"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507787    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f252f770-c18d-422d-b550-4157655338ba" containerName="registry"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507792    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="f252f770-c18d-422d-b550-4157655338ba" containerName="registry"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507798    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="init-config-reloader"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507803    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="init-config-reloader"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507808    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="alertmanager"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507812    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="alertmanager"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507821    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy-web"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507825    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy-web"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507830    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy-metric"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507835    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy-metric"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507874    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="prom-label-proxy"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507882    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="f252f770-c18d-422d-b550-4157655338ba" containerName="registry"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507888    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="cee18f1f-f19a-4d9c-bb3a-7bc5911463ac" containerName="console"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507892    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy-metric"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507898    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="config-reloader"
Jul 15 10:48:33.507896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507903    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy"
Jul 15 10:48:33.508449 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507910    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="0398c8b6-ae37-459a-aa5c-7f406c4ee98c" containerName="console"
Jul 15 10:48:33.508449 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507915    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="kube-rbac-proxy-web"
Jul 15 10:48:33.508449 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.507920    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e3722cc-84fb-4482-933d-7083120c61af" containerName="alertmanager"
Jul 15 10:48:33.512692 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.512678    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.515785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.515764    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t5pxl\""
Jul 15 10:48:33.515785 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.515779    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 15 10:48:33.515924 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.515789    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 15 10:48:33.515924 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.515801    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 15 10:48:33.515924 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.515808    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 15 10:48:33.515924 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.515874    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 15 10:48:33.515924 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.515809    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 15 10:48:33.515924 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.515765    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 15 10:48:33.516220 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.515939    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 15 10:48:33.522290 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.522258    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 15 10:48:33.527518 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.527498    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 10:48:33.642895 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.642834    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.642895 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.642879    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.643066 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.642915    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.643066 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.642958    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-config-volume\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.643066 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.642985    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-tls-assets\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.643066 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.643008    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.643066 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.643030    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.643066 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.643057    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.643341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.643079    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.643341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.643140    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.643341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.643169    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7crj9\" (UniqueName: \"kubernetes.io/projected/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-kube-api-access-7crj9\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.643341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.643193    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-web-config\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.643341 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.643256    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-config-out\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744411 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744372    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744532 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744434    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-config-volume\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744532 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744462    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-tls-assets\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744532 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744485    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744532 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744507    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744723 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744533    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744723 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744685    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744830 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744729    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744830 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744777    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7crj9\" (UniqueName: \"kubernetes.io/projected/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-kube-api-access-7crj9\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744830 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744807    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-web-config\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744979 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744839    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-config-out\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744979 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744865    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744979 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744922    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.744979 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.744944    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.745725 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.745700    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.747258 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.747226    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.747826 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.747629    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-config-volume\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.747826 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.747702    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.747826 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.747723    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-tls-assets\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.748034 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.747975    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.748161 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.748140    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-web-config\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.748344 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.748324    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.748562 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.748544    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.749264 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.749247    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-config-out\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.749653 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.749626    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.754503 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.754487    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7crj9\" (UniqueName: \"kubernetes.io/projected/80eb6dc6-c3c9-45e1-b979-5422a1f92c72-kube-api-access-7crj9\") pod \"alertmanager-main-0\" (UID: \"80eb6dc6-c3c9-45e1-b979-5422a1f92c72\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.822064 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.822043    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 10:48:33.958150 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:33.958128    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 10:48:33.960641 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:48:33.960616    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80eb6dc6_c3c9_45e1_b979_5422a1f92c72.slice/crio-a98954079fff3c091fc46792c066b15ee65810e7e270ec6dc894be63ad1e8bef WatchSource:0}: Error finding container a98954079fff3c091fc46792c066b15ee65810e7e270ec6dc894be63ad1e8bef: Status 404 returned error can't find the container with id a98954079fff3c091fc46792c066b15ee65810e7e270ec6dc894be63ad1e8bef
Jul 15 10:48:34.448433 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:34.448395    2536 generic.go:356] "Generic (PLEG): container finished" podID="80eb6dc6-c3c9-45e1-b979-5422a1f92c72" containerID="9a4b8c72b4073d2b98740cabd26ff715a4db0c40f350c0f4e97902cee96e93db" exitCode=0
Jul 15 10:48:34.448783 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:34.448481    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80eb6dc6-c3c9-45e1-b979-5422a1f92c72","Type":"ContainerDied","Data":"9a4b8c72b4073d2b98740cabd26ff715a4db0c40f350c0f4e97902cee96e93db"}
Jul 15 10:48:34.448783 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:34.448559    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80eb6dc6-c3c9-45e1-b979-5422a1f92c72","Type":"ContainerStarted","Data":"a98954079fff3c091fc46792c066b15ee65810e7e270ec6dc894be63ad1e8bef"}
Jul 15 10:48:34.594541 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:34.594516    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e3722cc-84fb-4482-933d-7083120c61af" path="/var/lib/kubelet/pods/6e3722cc-84fb-4482-933d-7083120c61af/volumes"
Jul 15 10:48:35.285727 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.285684    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-64555f4986-xjv87"]
Jul 15 10:48:35.289065 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.289049    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.291477 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.291457    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 15 10:48:35.291594 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.291546    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wjzf4\""
Jul 15 10:48:35.291594 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.291550    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 15 10:48:35.291913 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.291897    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 15 10:48:35.291976 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.291897    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 15 10:48:35.292044 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.291944    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 15 10:48:35.298455 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.298425    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 15 10:48:35.304300 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.304258    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-64555f4986-xjv87"]
Jul 15 10:48:35.455831 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.455796    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80eb6dc6-c3c9-45e1-b979-5422a1f92c72","Type":"ContainerStarted","Data":"5a4300bf075be7129e903a65c287c93fc88dfef0fa4ec04a09efffb269154f87"}
Jul 15 10:48:35.456137 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.455854    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80eb6dc6-c3c9-45e1-b979-5422a1f92c72","Type":"ContainerStarted","Data":"6a1df2c43a8b24cd3c119eaf671738d713a2cb6f96e625043af5cfdff364446c"}
Jul 15 10:48:35.456137 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.455881    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80eb6dc6-c3c9-45e1-b979-5422a1f92c72","Type":"ContainerStarted","Data":"6749c6bf2551b72a67df5b8ea3e0b02f994e75d4a79f1497c2588b6d5cbd7202"}
Jul 15 10:48:35.456137 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.455905    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80eb6dc6-c3c9-45e1-b979-5422a1f92c72","Type":"ContainerStarted","Data":"4f538e09a85b770a7d08089c661410b08e5c3482fd1f983809a791395b2e4b28"}
Jul 15 10:48:35.456137 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.455928    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80eb6dc6-c3c9-45e1-b979-5422a1f92c72","Type":"ContainerStarted","Data":"afd41540019ccdca60e57f27f338dff549669e1e67d85c083ba42c7d47a79dbf"}
Jul 15 10:48:35.456137 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.455953    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80eb6dc6-c3c9-45e1-b979-5422a1f92c72","Type":"ContainerStarted","Data":"98718f7675eac377e2cdf92a81c012417b7c30fc5f67c49f07c4a7698f5fdd16"}
Jul 15 10:48:35.459469 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.459449    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe873f8f-40ba-42db-9b07-99cf1d0562f4-serving-certs-ca-bundle\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.459523 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.459491    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-federate-client-tls\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.459560 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.459530    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe873f8f-40ba-42db-9b07-99cf1d0562f4-metrics-client-ca\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.459605 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.459567    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-telemeter-client-tls\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.459642 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.459625    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe873f8f-40ba-42db-9b07-99cf1d0562f4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.459686 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.459659    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5brw8\" (UniqueName: \"kubernetes.io/projected/fe873f8f-40ba-42db-9b07-99cf1d0562f4-kube-api-access-5brw8\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.459722 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.459689    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.459722 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.459711    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-secret-telemeter-client\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.483832 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.483779    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.469298331 podStartE2EDuration="2.483761099s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.462768ms" isStatefulPod=false podCreationTimestamp="2026-07-15 10:48:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:48:35.482174197 +0000 UTC m=+193.281647722" watchObservedRunningTime="2026-07-15 10:48:35.483761099 +0000 UTC m=+193.283234625"
Jul 15 10:48:35.560388 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.560324    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe873f8f-40ba-42db-9b07-99cf1d0562f4-metrics-client-ca\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.560388 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.560368    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-telemeter-client-tls\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.560569 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.560523    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe873f8f-40ba-42db-9b07-99cf1d0562f4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.560630 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.560581    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5brw8\" (UniqueName: \"kubernetes.io/projected/fe873f8f-40ba-42db-9b07-99cf1d0562f4-kube-api-access-5brw8\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.561173 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.560692    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.561173 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.560754    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-secret-telemeter-client\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.561173 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.560834    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe873f8f-40ba-42db-9b07-99cf1d0562f4-serving-certs-ca-bundle\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.561173 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.561035    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe873f8f-40ba-42db-9b07-99cf1d0562f4-metrics-client-ca\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.561173 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.561161    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-federate-client-tls\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.561549 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.561520    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe873f8f-40ba-42db-9b07-99cf1d0562f4-serving-certs-ca-bundle\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.561810 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.561792    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe873f8f-40ba-42db-9b07-99cf1d0562f4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.563011 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.562989    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-telemeter-client-tls\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.563538 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.563521    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-secret-telemeter-client\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.563907 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.563880    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.564060 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.564039    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fe873f8f-40ba-42db-9b07-99cf1d0562f4-federate-client-tls\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.569411 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.569394    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5brw8\" (UniqueName: \"kubernetes.io/projected/fe873f8f-40ba-42db-9b07-99cf1d0562f4-kube-api-access-5brw8\") pod \"telemeter-client-64555f4986-xjv87\" (UID: \"fe873f8f-40ba-42db-9b07-99cf1d0562f4\") " pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.586527 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.586504    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 10:48:35.586864 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.586836    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501" gracePeriod=600
Jul 15 10:48:35.586965 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.586860    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy-web" containerID="cri-o://d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564" gracePeriod=600
Jul 15 10:48:35.586965 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.586881    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="thanos-sidecar" containerID="cri-o://b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5" gracePeriod=600
Jul 15 10:48:35.586965 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.586888    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy" containerID="cri-o://60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897" gracePeriod=600
Jul 15 10:48:35.586965 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.586837    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="prometheus" containerID="cri-o://08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040" gracePeriod=600
Jul 15 10:48:35.587116 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.586913    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="config-reloader" containerID="cri-o://9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda" gracePeriod=600
Jul 15 10:48:35.598258 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.598235    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-64555f4986-xjv87"
Jul 15 10:48:35.734369 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:35.734343    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-64555f4986-xjv87"]
Jul 15 10:48:35.736374 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:48:35.736349    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe873f8f_40ba_42db_9b07_99cf1d0562f4.slice/crio-77f968a4546cbdcb1a9fd9129e1f9f9e80f94ea003736698931bc4a54191b682 WatchSource:0}: Error finding container 77f968a4546cbdcb1a9fd9129e1f9f9e80f94ea003736698931bc4a54191b682: Status 404 returned error can't find the container with id 77f968a4546cbdcb1a9fd9129e1f9f9e80f94ea003736698931bc4a54191b682
Jul 15 10:48:36.464439 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.464409    2536 generic.go:356] "Generic (PLEG): container finished" podID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerID="d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501" exitCode=0
Jul 15 10:48:36.464439 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.464433    2536 generic.go:356] "Generic (PLEG): container finished" podID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerID="60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897" exitCode=0
Jul 15 10:48:36.464439 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.464439    2536 generic.go:356] "Generic (PLEG): container finished" podID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerID="b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5" exitCode=0
Jul 15 10:48:36.464439 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.464447    2536 generic.go:356] "Generic (PLEG): container finished" podID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerID="9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda" exitCode=0
Jul 15 10:48:36.465006 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.464452    2536 generic.go:356] "Generic (PLEG): container finished" podID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerID="08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040" exitCode=0
Jul 15 10:48:36.465006 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.464480    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerDied","Data":"d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501"}
Jul 15 10:48:36.465006 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.464564    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerDied","Data":"60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897"}
Jul 15 10:48:36.465006 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.464611    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerDied","Data":"b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5"}
Jul 15 10:48:36.465006 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.464653    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerDied","Data":"9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda"}
Jul 15 10:48:36.465006 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.464692    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerDied","Data":"08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040"}
Jul 15 10:48:36.465568 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.465551    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64555f4986-xjv87" event={"ID":"fe873f8f-40ba-42db-9b07-99cf1d0562f4","Type":"ContainerStarted","Data":"77f968a4546cbdcb1a9fd9129e1f9f9e80f94ea003736698931bc4a54191b682"}
Jul 15 10:48:36.861829 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.861779    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:36.972304 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972251    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972487 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972316    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tmn8\" (UniqueName: \"kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-kube-api-access-6tmn8\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972487 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972351    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-0\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972487 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972377    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-config\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972487 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972412    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-metrics-client-certs\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972487 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972439    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-kubelet-serving-ca-bundle\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972487 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972465    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-thanos-prometheus-http-client-file\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972801 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972529    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-serving-certs-ca-bundle\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972801 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972555    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-metrics-client-ca\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972801 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972588    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-1\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972801 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972626    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-config-out\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972801 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972642    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-tls-assets\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972801 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972678    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-trusted-ca-bundle\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972801 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972708    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-2\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972801 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972737    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-tls\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972801 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972766    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-web-config\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.972801 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972788    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-grpc-tls\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.973310 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972817    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.973310 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972869    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-db\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.973310 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.972897    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-kube-rbac-proxy\") pod \"3b7e369c-7769-4450-ab38-ea7716ac457b\" (UID: \"3b7e369c-7769-4450-ab38-ea7716ac457b\") "
Jul 15 10:48:36.974862 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.973572    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:48:36.974862 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.974099    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:48:36.974862 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.974357    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:48:36.974862 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.974486    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:48:36.974862 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.974765    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:48:36.975259 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.974887    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:48:36.975259 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.975244    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 10:48:36.975391 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.975359    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:36.976207 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.976179    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:36.976952 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.976920    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 10:48:36.979730 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.979704    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:36.979832 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.979724    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-kube-api-access-6tmn8" (OuterVolumeSpecName: "kube-api-access-6tmn8") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "kube-api-access-6tmn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 10:48:36.979832 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.979790    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:36.979994 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.979940    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:36.980217 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.980182    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 10:48:36.980535 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.980502    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-config-out" (OuterVolumeSpecName: "config-out") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 10:48:36.980704 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.980670    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:36.980777 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.980740    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-config" (OuterVolumeSpecName: "config") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:36.981816 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.981794    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:36.990749 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:36.990723    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-web-config" (OuterVolumeSpecName: "web-config") pod "3b7e369c-7769-4450-ab38-ea7716ac457b" (UID: "3b7e369c-7769-4450-ab38-ea7716ac457b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 10:48:37.074424 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074390    2536 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074424 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074419    2536 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074448    2536 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-web-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074458    2536 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-grpc-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074469    2536 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074484    2536 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-db\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074498    2536 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-kube-rbac-proxy\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074512    2536 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074526    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tmn8\" (UniqueName: \"kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-kube-api-access-6tmn8\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074542    2536 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074551    2536 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-config\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074559    2536 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-secret-metrics-client-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074568    2536 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074577    2536 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3b7e369c-7769-4450-ab38-ea7716ac457b-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074586    2536 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074595    2536 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-configmap-metrics-client-ca\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074604    2536 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.074607 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074613    2536 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3b7e369c-7769-4450-ab38-ea7716ac457b-config-out\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.075132 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074623    2536 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3b7e369c-7769-4450-ab38-ea7716ac457b-tls-assets\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.075132 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.074631    2536 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7e369c-7769-4450-ab38-ea7716ac457b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 10:48:37.471423 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.471396    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64555f4986-xjv87" event={"ID":"fe873f8f-40ba-42db-9b07-99cf1d0562f4","Type":"ContainerStarted","Data":"2b713c9b61e37067c1f1b609f8b27d34a3cf7639c1168bdc1af79d3ade2a3e8c"}
Jul 15 10:48:37.474423 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.474399    2536 generic.go:356] "Generic (PLEG): container finished" podID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerID="d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564" exitCode=0
Jul 15 10:48:37.474512 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.474445    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerDied","Data":"d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564"}
Jul 15 10:48:37.474512 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.474478    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.474512 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.474495    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3b7e369c-7769-4450-ab38-ea7716ac457b","Type":"ContainerDied","Data":"754ae51e15ca161af88afad05d8492c6f1434e184beed244d7d8263937909077"}
Jul 15 10:48:37.474713 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.474526    2536 scope.go:117] "RemoveContainer" containerID="d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501"
Jul 15 10:48:37.482705 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.482686    2536 scope.go:117] "RemoveContainer" containerID="60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897"
Jul 15 10:48:37.490727 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.490708    2536 scope.go:117] "RemoveContainer" containerID="d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564"
Jul 15 10:48:37.497992 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.497974    2536 scope.go:117] "RemoveContainer" containerID="b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5"
Jul 15 10:48:37.499940 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.499916    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 10:48:37.503395 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.503372    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 10:48:37.511700 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.511684    2536 scope.go:117] "RemoveContainer" containerID="9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda"
Jul 15 10:48:37.526668 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.526612    2536 scope.go:117] "RemoveContainer" containerID="08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040"
Jul 15 10:48:37.528222 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528200    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 10:48:37.528556 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528540    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="thanos-sidecar"
Jul 15 10:48:37.528638 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528559    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="thanos-sidecar"
Jul 15 10:48:37.528638 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528585    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy-thanos"
Jul 15 10:48:37.528638 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528594    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy-thanos"
Jul 15 10:48:37.528638 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528607    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy"
Jul 15 10:48:37.528638 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528616    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy"
Jul 15 10:48:37.528638 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528629    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="init-config-reloader"
Jul 15 10:48:37.528638 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528637    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="init-config-reloader"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528646    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="config-reloader"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528654    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="config-reloader"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528663    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy-web"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528671    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy-web"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528684    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="prometheus"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528693    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="prometheus"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528765    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy-web"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528778    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="config-reloader"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528788    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy-thanos"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528799    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="prometheus"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528809    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="thanos-sidecar"
Jul 15 10:48:37.529012 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.528817    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" containerName="kube-rbac-proxy"
Jul 15 10:48:37.533855 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.533836    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.536495 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.536478    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 10:48:37.536582 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.536478    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 10:48:37.536670 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.536655    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 10:48:37.536712 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.536696    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 10:48:37.536753 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.536745    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 10:48:37.536907 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.536893    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 10:48:37.537189 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.537172    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 10:48:37.537231 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.537172    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 10:48:37.538486 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.538464    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 10:48:37.538589 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.538486    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-g2kjz\""
Jul 15 10:48:37.538589 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.538515    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 10:48:37.538839 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.538803    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 10:48:37.538913 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.538825    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 10:48:37.538913 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.538890    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7m26jock92p8b\""
Jul 15 10:48:37.539285 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.539255    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 10:48:37.541957 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.541940    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 10:48:37.544036 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.544019    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 10:48:37.548752 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.548702    2536 scope.go:117] "RemoveContainer" containerID="f7feb8a8a6ac4a66e6962e900655d611ca50ba5dfba59b160e8a92c556701aad"
Jul 15 10:48:37.550261 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.550162    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 10:48:37.556983 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.556960    2536 scope.go:117] "RemoveContainer" containerID="d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501"
Jul 15 10:48:37.557235 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:37.557216    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501\": container with ID starting with d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501 not found: ID does not exist" containerID="d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501"
Jul 15 10:48:37.557343 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.557241    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501"} err="failed to get container status \"d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501\": rpc error: code = NotFound desc = could not find container \"d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501\": container with ID starting with d3b14d14b69b04d74b6c4f1da629bc4725332010a4aca08ddc3a5970f5271501 not found: ID does not exist"
Jul 15 10:48:37.557343 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.557260    2536 scope.go:117] "RemoveContainer" containerID="60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897"
Jul 15 10:48:37.557522 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:37.557501    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897\": container with ID starting with 60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897 not found: ID does not exist" containerID="60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897"
Jul 15 10:48:37.557580 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.557526    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897"} err="failed to get container status \"60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897\": rpc error: code = NotFound desc = could not find container \"60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897\": container with ID starting with 60af44dfd6b038b588d95320c9f191e4e55845f02365724200bb9f2498e06897 not found: ID does not exist"
Jul 15 10:48:37.557580 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.557548    2536 scope.go:117] "RemoveContainer" containerID="d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564"
Jul 15 10:48:37.557810 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:37.557791    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564\": container with ID starting with d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564 not found: ID does not exist" containerID="d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564"
Jul 15 10:48:37.557896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.557812    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564"} err="failed to get container status \"d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564\": rpc error: code = NotFound desc = could not find container \"d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564\": container with ID starting with d52df2153088bf70f2b4dc61faacab8736c0f2d56d61c309616425445a2f2564 not found: ID does not exist"
Jul 15 10:48:37.557896 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.557825    2536 scope.go:117] "RemoveContainer" containerID="b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5"
Jul 15 10:48:37.558091 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:37.558073    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5\": container with ID starting with b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5 not found: ID does not exist" containerID="b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5"
Jul 15 10:48:37.558156 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.558096    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5"} err="failed to get container status \"b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5\": rpc error: code = NotFound desc = could not find container \"b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5\": container with ID starting with b459850d48510b355837f3e212eb70348dfc7addab3e60a63aa657ea863c95d5 not found: ID does not exist"
Jul 15 10:48:37.558156 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.558112    2536 scope.go:117] "RemoveContainer" containerID="9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda"
Jul 15 10:48:37.558381 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:37.558363    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda\": container with ID starting with 9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda not found: ID does not exist" containerID="9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda"
Jul 15 10:48:37.558467 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.558384    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda"} err="failed to get container status \"9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda\": rpc error: code = NotFound desc = could not find container \"9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda\": container with ID starting with 9bec9debc4ce1d3fd324d56e868092be916b872c14a2ccbfa31bda1f7ae7dbda not found: ID does not exist"
Jul 15 10:48:37.558467 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.558397    2536 scope.go:117] "RemoveContainer" containerID="08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040"
Jul 15 10:48:37.558630 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:37.558609    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040\": container with ID starting with 08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040 not found: ID does not exist" containerID="08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040"
Jul 15 10:48:37.558758 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.558636    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040"} err="failed to get container status \"08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040\": rpc error: code = NotFound desc = could not find container \"08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040\": container with ID starting with 08c5653a2f82461a7157db3aaf3d2744f05aa6bec43504230fd49af18cece040 not found: ID does not exist"
Jul 15 10:48:37.558758 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.558656    2536 scope.go:117] "RemoveContainer" containerID="f7feb8a8a6ac4a66e6962e900655d611ca50ba5dfba59b160e8a92c556701aad"
Jul 15 10:48:37.558921 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:48:37.558903    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7feb8a8a6ac4a66e6962e900655d611ca50ba5dfba59b160e8a92c556701aad\": container with ID starting with f7feb8a8a6ac4a66e6962e900655d611ca50ba5dfba59b160e8a92c556701aad not found: ID does not exist" containerID="f7feb8a8a6ac4a66e6962e900655d611ca50ba5dfba59b160e8a92c556701aad"
Jul 15 10:48:37.558979 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.558923    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7feb8a8a6ac4a66e6962e900655d611ca50ba5dfba59b160e8a92c556701aad"} err="failed to get container status \"f7feb8a8a6ac4a66e6962e900655d611ca50ba5dfba59b160e8a92c556701aad\": rpc error: code = NotFound desc = could not find container \"f7feb8a8a6ac4a66e6962e900655d611ca50ba5dfba59b160e8a92c556701aad\": container with ID starting with f7feb8a8a6ac4a66e6962e900655d611ca50ba5dfba59b160e8a92c556701aad not found: ID does not exist"
Jul 15 10:48:37.680443 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680369    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-config-out\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680443 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680431    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680669 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680454    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78mlh\" (UniqueName: \"kubernetes.io/projected/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-kube-api-access-78mlh\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680669 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680494    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680669 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680536    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680669 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680558    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680669 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680629    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680669 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680654    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680857 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680705    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680857 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680721    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680857 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680737    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680857 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680773    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680857 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680799    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680857 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680814    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680857 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680828    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.680857 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680843    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.681083 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680895    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.681083 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680929    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.681083 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.680973    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-config\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.681083 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.681003    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-web-config\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.781791 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.781765    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.781791 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.781791    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78mlh\" (UniqueName: \"kubernetes.io/projected/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-kube-api-access-78mlh\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.781941 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.781810    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.781941 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.781831    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.781941 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.781847    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.781941 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.781884    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.781941 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.781906    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.781941 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.781940    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.781960    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.781982    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782012    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782035    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782058    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782083    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782105    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782132    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782155    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782178    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-config\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782230 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782200    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-web-config\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782973 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782245    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-config-out\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782973 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782572    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782973 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782609    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.782973 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.782655    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.783451 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.783411    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.783586 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.783560    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.783647 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.783613    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.785121 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.785091    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-config-out\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.785247 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.785221    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.786406 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.786183    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.786406 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.786341    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.786406 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.786345    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-config\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.786406 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.786394    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.786661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.786469    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.787005 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.786980    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.787395 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.787368    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.787708 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.787687    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-web-config\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.787791 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.787756    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.788509 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.788488    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.788901 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.788878    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.790590 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.790569    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78mlh\" (UniqueName: \"kubernetes.io/projected/bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09-kube-api-access-78mlh\") pod \"prometheus-k8s-0\" (UID: \"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.854017 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.853994    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:48:37.998334 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:37.998299    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 10:48:38.003624 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:48:38.003587    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbddbe3a4_0f70_4a0e_ad4d_d21b1e40de09.slice/crio-c25605b90828ee30df170030dc11ff4161d64432472f3f4f0289cddb0edf3584 WatchSource:0}: Error finding container c25605b90828ee30df170030dc11ff4161d64432472f3f4f0289cddb0edf3584: Status 404 returned error can't find the container with id c25605b90828ee30df170030dc11ff4161d64432472f3f4f0289cddb0edf3584
Jul 15 10:48:38.480015 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:38.479977    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64555f4986-xjv87" event={"ID":"fe873f8f-40ba-42db-9b07-99cf1d0562f4","Type":"ContainerStarted","Data":"c552e1e1cf11e0512b95cd89ddfb0a958e745d55a547574ade6fd69047bd1692"}
Jul 15 10:48:38.480474 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:38.480047    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-64555f4986-xjv87" event={"ID":"fe873f8f-40ba-42db-9b07-99cf1d0562f4","Type":"ContainerStarted","Data":"bf5d09bbfce58ea4cf7fbade11ae43f19be4a0decc9a71d94f88fc3c9dc90cfe"}
Jul 15 10:48:38.481387 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:38.481361    2536 generic.go:356] "Generic (PLEG): container finished" podID="bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09" containerID="332fc69445605206e4efc8f33d21946582d764273da873cd8dd746cd313a0e37" exitCode=0
Jul 15 10:48:38.481494 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:38.481451    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09","Type":"ContainerDied","Data":"332fc69445605206e4efc8f33d21946582d764273da873cd8dd746cd313a0e37"}
Jul 15 10:48:38.481554 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:38.481507    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09","Type":"ContainerStarted","Data":"c25605b90828ee30df170030dc11ff4161d64432472f3f4f0289cddb0edf3584"}
Jul 15 10:48:38.506121 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:38.506064    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-64555f4986-xjv87" podStartSLOduration=1.886353183 podStartE2EDuration="3.506044527s" totalImagesPullingTime="1.619691344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:48:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:48:38.504235732 +0000 UTC m=+196.303709257" watchObservedRunningTime="2026-07-15 10:48:38.506044527 +0000 UTC m=+196.305518053"
Jul 15 10:48:38.596949 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:38.596922    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b7e369c-7769-4450-ab38-ea7716ac457b" path="/var/lib/kubelet/pods/3b7e369c-7769-4450-ab38-ea7716ac457b/volumes"
Jul 15 10:48:39.490914 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:39.490882    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09","Type":"ContainerStarted","Data":"f4945c517e1eec688ec21030039a95401e27886d1684036a622d2d049fe563d2"}
Jul 15 10:48:39.491305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:39.490950    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09","Type":"ContainerStarted","Data":"9a94441f16b83e972c4f46395962dd8146bfbb222d8e898adf4624e54d734116"}
Jul 15 10:48:39.491305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:39.490977    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09","Type":"ContainerStarted","Data":"731e54d950c9e53ac10914dc4cc83410bab12f4204b9cb44acef5d67e7f84014"}
Jul 15 10:48:39.491305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:39.490999    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09","Type":"ContainerStarted","Data":"ee64f0f10e7650d27dc6d6e0e5acd79966707a6cbb7120d61bd3fd6c09dbc713"}
Jul 15 10:48:39.491305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:39.491025    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09","Type":"ContainerStarted","Data":"a2f82a7c1600f5d74a33bf85b53bcbe13bcd7cea8bffc0b460a07ab80dbd0501"}
Jul 15 10:48:39.491305 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:39.491047    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09","Type":"ContainerStarted","Data":"49d4adc16ddd2c447e204480c10f8830d2d0d4a84397ccb1875a19d5e2587489"}
Jul 15 10:48:39.544628 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:39.544563    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.52650866 podStartE2EDuration="2.544543581s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.034921ms" isStatefulPod=false podCreationTimestamp="2026-07-15 10:48:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:48:39.540121859 +0000 UTC m=+197.339595384" watchObservedRunningTime="2026-07-15 10:48:39.544543581 +0000 UTC m=+197.344017105"
Jul 15 10:48:42.854650 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:48:42.854603    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:49:37.854370 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:49:37.854328    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:49:37.870062 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:49:37.870038    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:49:38.723478 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:49:38.723451    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 10:50:22.515228 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:50:22.515204    2536 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 10:53:43.800074 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.800037    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bbbfb8bbc-d2cf8"]
Jul 15 10:53:43.803227 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.803210    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:43.805639 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.805610    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 10:53:43.805639 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.805626    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 10:53:43.805825 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.805629    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 10:53:43.805825 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.805626    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 10:53:43.806806 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.806786    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f77v6\""
Jul 15 10:53:43.806894 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.806836    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 10:53:43.811325 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.811306    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 10:53:43.814046 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.814025    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bbbfb8bbc-d2cf8"]
Jul 15 10:53:43.907425 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.907403    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6220d036-3421-4865-8c0e-33a51551ec79-console-serving-cert\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:43.907548 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.907430    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kp5x\" (UniqueName: \"kubernetes.io/projected/6220d036-3421-4865-8c0e-33a51551ec79-kube-api-access-2kp5x\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:43.907548 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.907462    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-console-config\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:43.907548 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.907480    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-trusted-ca-bundle\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:43.907661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.907550    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-oauth-serving-cert\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:43.907661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.907597    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-service-ca\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:43.907661 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:43.907631    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6220d036-3421-4865-8c0e-33a51551ec79-console-oauth-config\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.007920 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.007895    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-service-ca\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.008018 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.007931    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6220d036-3421-4865-8c0e-33a51551ec79-console-oauth-config\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.008018 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.007969    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6220d036-3421-4865-8c0e-33a51551ec79-console-serving-cert\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.008018 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.007991    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kp5x\" (UniqueName: \"kubernetes.io/projected/6220d036-3421-4865-8c0e-33a51551ec79-kube-api-access-2kp5x\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.008147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.008040    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-console-config\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.008147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.008066    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-trusted-ca-bundle\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.008147 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.008087    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-oauth-serving-cert\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.008720 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.008650    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-service-ca\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.008943 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.008776    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-oauth-serving-cert\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.008943 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.008884    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-console-config\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.009140 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.009117    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6220d036-3421-4865-8c0e-33a51551ec79-trusted-ca-bundle\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.010725 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.010699    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6220d036-3421-4865-8c0e-33a51551ec79-console-serving-cert\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.010804 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.010699    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6220d036-3421-4865-8c0e-33a51551ec79-console-oauth-config\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.016122 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.016100    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kp5x\" (UniqueName: \"kubernetes.io/projected/6220d036-3421-4865-8c0e-33a51551ec79-kube-api-access-2kp5x\") pod \"console-5bbbfb8bbc-d2cf8\" (UID: \"6220d036-3421-4865-8c0e-33a51551ec79\") " pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.112971 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.112917    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:44.241021 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.240932    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bbbfb8bbc-d2cf8"]
Jul 15 10:53:44.243174 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:53:44.243149    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6220d036_3421_4865_8c0e_33a51551ec79.slice/crio-31442947a59c2135e1d9da6fdf507ef2c8047ebbeeeb84bf0253e68f4901da3c WatchSource:0}: Error finding container 31442947a59c2135e1d9da6fdf507ef2c8047ebbeeeb84bf0253e68f4901da3c: Status 404 returned error can't find the container with id 31442947a59c2135e1d9da6fdf507ef2c8047ebbeeeb84bf0253e68f4901da3c
Jul 15 10:53:44.603406 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.603372    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bbbfb8bbc-d2cf8" event={"ID":"6220d036-3421-4865-8c0e-33a51551ec79","Type":"ContainerStarted","Data":"27aa9657dac2b38ba732aa4f63059a3f6c0e63c8229034f88502447c70a9888e"}
Jul 15 10:53:44.603565 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.603419    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bbbfb8bbc-d2cf8" event={"ID":"6220d036-3421-4865-8c0e-33a51551ec79","Type":"ContainerStarted","Data":"31442947a59c2135e1d9da6fdf507ef2c8047ebbeeeb84bf0253e68f4901da3c"}
Jul 15 10:53:44.622811 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:44.622764    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bbbfb8bbc-d2cf8" podStartSLOduration=1.622750619 podStartE2EDuration="1.622750619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:53:44.621822002 +0000 UTC m=+502.421295530" watchObservedRunningTime="2026-07-15 10:53:44.622750619 +0000 UTC m=+502.422224144"
Jul 15 10:53:52.037261 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.037227    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fnjp7"]
Jul 15 10:53:52.040885 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.040867    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7"
Jul 15 10:53:52.044231 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.044202    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 15 10:53:52.044371 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.044260    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7tcds\""
Jul 15 10:53:52.044371 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.044293    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 15 10:53:52.050497 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.050476    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fnjp7"]
Jul 15 10:53:52.173393 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.173368    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckddx\" (UniqueName: \"kubernetes.io/projected/98f2efaa-8ed9-46fc-af52-33bbb4edf210-kube-api-access-ckddx\") pod \"cert-manager-webhook-6486ffd997-fnjp7\" (UID: \"98f2efaa-8ed9-46fc-af52-33bbb4edf210\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7"
Jul 15 10:53:52.173500 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.173429    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98f2efaa-8ed9-46fc-af52-33bbb4edf210-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fnjp7\" (UID: \"98f2efaa-8ed9-46fc-af52-33bbb4edf210\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7"
Jul 15 10:53:52.274248 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.274223    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98f2efaa-8ed9-46fc-af52-33bbb4edf210-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fnjp7\" (UID: \"98f2efaa-8ed9-46fc-af52-33bbb4edf210\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7"
Jul 15 10:53:52.274428 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.274266    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckddx\" (UniqueName: \"kubernetes.io/projected/98f2efaa-8ed9-46fc-af52-33bbb4edf210-kube-api-access-ckddx\") pod \"cert-manager-webhook-6486ffd997-fnjp7\" (UID: \"98f2efaa-8ed9-46fc-af52-33bbb4edf210\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7"
Jul 15 10:53:52.282989 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.282964    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98f2efaa-8ed9-46fc-af52-33bbb4edf210-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fnjp7\" (UID: \"98f2efaa-8ed9-46fc-af52-33bbb4edf210\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7"
Jul 15 10:53:52.283152 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.283136    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckddx\" (UniqueName: \"kubernetes.io/projected/98f2efaa-8ed9-46fc-af52-33bbb4edf210-kube-api-access-ckddx\") pod \"cert-manager-webhook-6486ffd997-fnjp7\" (UID: \"98f2efaa-8ed9-46fc-af52-33bbb4edf210\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7"
Jul 15 10:53:52.350023 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.349974    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7"
Jul 15 10:53:52.481058 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.481025    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fnjp7"]
Jul 15 10:53:52.483961 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:53:52.483937    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98f2efaa_8ed9_46fc_af52_33bbb4edf210.slice/crio-4448db1c0fc7e7a5dc4e1ca595e1320974dc003d2b342c7491062d9dbf9d6159 WatchSource:0}: Error finding container 4448db1c0fc7e7a5dc4e1ca595e1320974dc003d2b342c7491062d9dbf9d6159: Status 404 returned error can't find the container with id 4448db1c0fc7e7a5dc4e1ca595e1320974dc003d2b342c7491062d9dbf9d6159
Jul 15 10:53:52.485987 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.485965    2536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 10:53:52.633145 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:52.633082    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7" event={"ID":"98f2efaa-8ed9-46fc-af52-33bbb4edf210","Type":"ContainerStarted","Data":"4448db1c0fc7e7a5dc4e1ca595e1320974dc003d2b342c7491062d9dbf9d6159"}
Jul 15 10:53:54.113911 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.113879    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:54.114394 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.113950    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:54.120414 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.120395    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:54.454941 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.454861    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9"]
Jul 15 10:53:54.461923 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.461895    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9"
Jul 15 10:53:54.464567 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.464482    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lpknx\""
Jul 15 10:53:54.469647 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.469180    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9"]
Jul 15 10:53:54.595812 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.595773    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ljdw9\" (UID: \"d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9"
Jul 15 10:53:54.596005 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.595823    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg6kx\" (UniqueName: \"kubernetes.io/projected/d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7-kube-api-access-jg6kx\") pod \"cert-manager-cainjector-559cdd8c87-ljdw9\" (UID: \"d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9"
Jul 15 10:53:54.648778 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.648747    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bbbfb8bbc-d2cf8"
Jul 15 10:53:54.696714 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.696680    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ljdw9\" (UID: \"d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9"
Jul 15 10:53:54.696861 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.696725    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg6kx\" (UniqueName: \"kubernetes.io/projected/d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7-kube-api-access-jg6kx\") pod \"cert-manager-cainjector-559cdd8c87-ljdw9\" (UID: \"d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9"
Jul 15 10:53:54.719466 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.719414    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg6kx\" (UniqueName: \"kubernetes.io/projected/d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7-kube-api-access-jg6kx\") pod \"cert-manager-cainjector-559cdd8c87-ljdw9\" (UID: \"d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9"
Jul 15 10:53:54.720267 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.720240    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ljdw9\" (UID: \"d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9"
Jul 15 10:53:54.775219 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:54.775181    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9"
Jul 15 10:53:56.065140 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:56.065116    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9"]
Jul 15 10:53:56.066796 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:53:56.066771    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9dfa07d_c6ff_443c_b8d6_ed6cd2227ce7.slice/crio-5583b6fd11ab9fb9d07b8ccee5aa55b26b10ac9f74e996f6bb7b83afb6e39ff3 WatchSource:0}: Error finding container 5583b6fd11ab9fb9d07b8ccee5aa55b26b10ac9f74e996f6bb7b83afb6e39ff3: Status 404 returned error can't find the container with id 5583b6fd11ab9fb9d07b8ccee5aa55b26b10ac9f74e996f6bb7b83afb6e39ff3
Jul 15 10:53:56.653340 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:56.653294    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7" event={"ID":"98f2efaa-8ed9-46fc-af52-33bbb4edf210","Type":"ContainerStarted","Data":"d4e6d0c77b3e0dad02b276bb6a7bd7443cf2226c1e4b419a13914207fd6b4580"}
Jul 15 10:53:56.653509 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:56.653459    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7"
Jul 15 10:53:56.654812 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:56.654789    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9" event={"ID":"d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7","Type":"ContainerStarted","Data":"b821e736daecaffbb964d69050d7014921515e0245a155e93c7acb339231edc3"}
Jul 15 10:53:56.654812 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:56.654820    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9" event={"ID":"d9dfa07d-c6ff-443c-b8d6-ed6cd2227ce7","Type":"ContainerStarted","Data":"5583b6fd11ab9fb9d07b8ccee5aa55b26b10ac9f74e996f6bb7b83afb6e39ff3"}
Jul 15 10:53:56.676267 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:56.676222    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7" podStartSLOduration=1.160278087 podStartE2EDuration="4.676209769s" totalImagesPullingTime="3.515931682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:53:56.675424349 +0000 UTC m=+514.474897875" watchObservedRunningTime="2026-07-15 10:53:56.676209769 +0000 UTC m=+514.475683293"
Jul 15 10:53:56.692101 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:53:56.692062    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ljdw9" podStartSLOduration=2.692050117 podStartE2EDuration="2.692050117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:53:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:53:56.691140982 +0000 UTC m=+514.490614508" watchObservedRunningTime="2026-07-15 10:53:56.692050117 +0000 UTC m=+514.491523641"
Jul 15 10:54:02.660367 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:02.660335    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fnjp7"
Jul 15 10:54:07.840578 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:07.840544    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr"]
Jul 15 10:54:07.846817 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:07.846797    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr"
Jul 15 10:54:07.849412 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:07.849392    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 10:54:07.850220 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:07.850190    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-p9j5h\""
Jul 15 10:54:07.850340 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:07.850262    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 10:54:07.855624 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:07.855602    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr"]
Jul 15 10:54:07.990059 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:07.990031    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f73f68b-76a5-4ece-9bf1-1af8efaf49e0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5mzxr\" (UID: \"3f73f68b-76a5-4ece-9bf1-1af8efaf49e0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr"
Jul 15 10:54:07.990192 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:07.990068    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rckm\" (UniqueName: \"kubernetes.io/projected/3f73f68b-76a5-4ece-9bf1-1af8efaf49e0-kube-api-access-6rckm\") pod \"openshift-lws-operator-fd8ccff4c-5mzxr\" (UID: \"3f73f68b-76a5-4ece-9bf1-1af8efaf49e0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr"
Jul 15 10:54:08.090449 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:08.090423    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f73f68b-76a5-4ece-9bf1-1af8efaf49e0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5mzxr\" (UID: \"3f73f68b-76a5-4ece-9bf1-1af8efaf49e0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr"
Jul 15 10:54:08.090548 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:08.090461    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rckm\" (UniqueName: \"kubernetes.io/projected/3f73f68b-76a5-4ece-9bf1-1af8efaf49e0-kube-api-access-6rckm\") pod \"openshift-lws-operator-fd8ccff4c-5mzxr\" (UID: \"3f73f68b-76a5-4ece-9bf1-1af8efaf49e0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr"
Jul 15 10:54:08.090867 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:08.090827    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f73f68b-76a5-4ece-9bf1-1af8efaf49e0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5mzxr\" (UID: \"3f73f68b-76a5-4ece-9bf1-1af8efaf49e0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr"
Jul 15 10:54:08.099606 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:08.099580    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rckm\" (UniqueName: \"kubernetes.io/projected/3f73f68b-76a5-4ece-9bf1-1af8efaf49e0-kube-api-access-6rckm\") pod \"openshift-lws-operator-fd8ccff4c-5mzxr\" (UID: \"3f73f68b-76a5-4ece-9bf1-1af8efaf49e0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr"
Jul 15 10:54:08.156586 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:08.156562    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr"
Jul 15 10:54:08.287196 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:08.287168    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr"]
Jul 15 10:54:08.289723 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:54:08.289690    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f73f68b_76a5_4ece_9bf1_1af8efaf49e0.slice/crio-2960590fe6339bdcf797c57fe276c877bb884fdb91b7f44fb1f5ab1e051aa50c WatchSource:0}: Error finding container 2960590fe6339bdcf797c57fe276c877bb884fdb91b7f44fb1f5ab1e051aa50c: Status 404 returned error can't find the container with id 2960590fe6339bdcf797c57fe276c877bb884fdb91b7f44fb1f5ab1e051aa50c
Jul 15 10:54:08.701018 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:08.700982    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr" event={"ID":"3f73f68b-76a5-4ece-9bf1-1af8efaf49e0","Type":"ContainerStarted","Data":"2960590fe6339bdcf797c57fe276c877bb884fdb91b7f44fb1f5ab1e051aa50c"}
Jul 15 10:54:11.714816 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:11.714780    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr" event={"ID":"3f73f68b-76a5-4ece-9bf1-1af8efaf49e0","Type":"ContainerStarted","Data":"5d455f1cf536930c4f53613b0687ef489b62b5f373fb2f46c7e1160e811b9380"}
Jul 15 10:54:11.731112 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:11.731063    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5mzxr" podStartSLOduration=2.32410208 podStartE2EDuration="4.731048096s" totalImagesPullingTime="2.406946016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:54:11.729953501 +0000 UTC m=+529.529427027" watchObservedRunningTime="2026-07-15 10:54:11.731048096 +0000 UTC m=+529.530521621"
Jul 15 10:54:31.153290 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.153256    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"]
Jul 15 10:54:31.156770 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.156754    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.159789 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.159771    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hpbg6\""
Jul 15 10:54:31.160633 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.160615    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 15 10:54:31.160727 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.160668    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 15 10:54:31.160816 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.160803    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 15 10:54:31.178989 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.178971    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"]
Jul 15 10:54:31.260367 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.260335    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9119c75d-106d-4de2-b099-2f1088e0a199-metrics-cert\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.260367 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.260365    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbg4x\" (UniqueName: \"kubernetes.io/projected/9119c75d-106d-4de2-b099-2f1088e0a199-kube-api-access-cbg4x\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.260527 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.260389    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9119c75d-106d-4de2-b099-2f1088e0a199-cert\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.260572 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.260547    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9119c75d-106d-4de2-b099-2f1088e0a199-manager-config\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.361647 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.361622    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9119c75d-106d-4de2-b099-2f1088e0a199-manager-config\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.361760 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.361663    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9119c75d-106d-4de2-b099-2f1088e0a199-metrics-cert\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.361760 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.361681    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbg4x\" (UniqueName: \"kubernetes.io/projected/9119c75d-106d-4de2-b099-2f1088e0a199-kube-api-access-cbg4x\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.361760 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.361723    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9119c75d-106d-4de2-b099-2f1088e0a199-cert\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.362246 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.362226    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9119c75d-106d-4de2-b099-2f1088e0a199-manager-config\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.364197 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.364174    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9119c75d-106d-4de2-b099-2f1088e0a199-metrics-cert\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.364292 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.364219    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9119c75d-106d-4de2-b099-2f1088e0a199-cert\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.369965 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.369947    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbg4x\" (UniqueName: \"kubernetes.io/projected/9119c75d-106d-4de2-b099-2f1088e0a199-kube-api-access-cbg4x\") pod \"lws-controller-manager-6d9b94dcd4-4tnld\" (UID: \"9119c75d-106d-4de2-b099-2f1088e0a199\") " pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.466552 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.466531    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:31.619312 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.619265    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"]
Jul 15 10:54:31.622732 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:54:31.622707    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9119c75d_106d_4de2_b099_2f1088e0a199.slice/crio-c40089cdacd11bde3ef8ab305e383b31192aa21659608387aac27abb559edd15 WatchSource:0}: Error finding container c40089cdacd11bde3ef8ab305e383b31192aa21659608387aac27abb559edd15: Status 404 returned error can't find the container with id c40089cdacd11bde3ef8ab305e383b31192aa21659608387aac27abb559edd15
Jul 15 10:54:31.793695 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:31.793625    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld" event={"ID":"9119c75d-106d-4de2-b099-2f1088e0a199","Type":"ContainerStarted","Data":"c40089cdacd11bde3ef8ab305e383b31192aa21659608387aac27abb559edd15"}
Jul 15 10:54:33.803806 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:33.803771    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld" event={"ID":"9119c75d-106d-4de2-b099-2f1088e0a199","Type":"ContainerStarted","Data":"4cc07a96242344ea55fa5a0b3d0d076937e3d50f5f838d7864c8228d8af0c801"}
Jul 15 10:54:33.804193 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:33.803899    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:54:33.823065 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:33.822929    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld" podStartSLOduration=1.398367439 podStartE2EDuration="2.822912921s" totalImagesPullingTime="1.424545482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:54:33.822261894 +0000 UTC m=+551.621735418" watchObservedRunningTime="2026-07-15 10:54:33.822912921 +0000 UTC m=+551.622386475"
Jul 15 10:54:44.810089 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:54:44.810062    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d9b94dcd4-4tnld"
Jul 15 10:55:20.009783 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.009748    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"]
Jul 15 10:55:20.013310 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.013269    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"
Jul 15 10:55:20.015648 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.015626    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 10:55:20.015648 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.015642    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7z7qw\""
Jul 15 10:55:20.015791 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.015628    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 10:55:20.028121 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.028103    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"]
Jul 15 10:55:20.126572 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.126539    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggrw8\" (UniqueName: \"kubernetes.io/projected/c0f17d07-73e9-4bab-b471-977e8074167f-kube-api-access-ggrw8\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-frhbf\" (UID: \"c0f17d07-73e9-4bab-b471-977e8074167f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"
Jul 15 10:55:20.126572 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.126576    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0f17d07-73e9-4bab-b471-977e8074167f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-frhbf\" (UID: \"c0f17d07-73e9-4bab-b471-977e8074167f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"
Jul 15 10:55:20.227768 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.227726    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggrw8\" (UniqueName: \"kubernetes.io/projected/c0f17d07-73e9-4bab-b471-977e8074167f-kube-api-access-ggrw8\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-frhbf\" (UID: \"c0f17d07-73e9-4bab-b471-977e8074167f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"
Jul 15 10:55:20.227906 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.227777    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0f17d07-73e9-4bab-b471-977e8074167f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-frhbf\" (UID: \"c0f17d07-73e9-4bab-b471-977e8074167f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"
Jul 15 10:55:20.228145 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.228125    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0f17d07-73e9-4bab-b471-977e8074167f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-frhbf\" (UID: \"c0f17d07-73e9-4bab-b471-977e8074167f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"
Jul 15 10:55:20.236880 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.236855    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggrw8\" (UniqueName: \"kubernetes.io/projected/c0f17d07-73e9-4bab-b471-977e8074167f-kube-api-access-ggrw8\") pod \"kuadrant-operator-controller-manager-bb7ddb7d7-frhbf\" (UID: \"c0f17d07-73e9-4bab-b471-977e8074167f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"
Jul 15 10:55:20.322873 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.322820    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"
Jul 15 10:55:20.464133 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.464107    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"]
Jul 15 10:55:20.466749 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:55:20.466720    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f17d07_73e9_4bab_b471_977e8074167f.slice/crio-ce633f4010f5a47363a9e999f232518cda87f5a33e05abc9afeb6aa6c0df9aca WatchSource:0}: Error finding container ce633f4010f5a47363a9e999f232518cda87f5a33e05abc9afeb6aa6c0df9aca: Status 404 returned error can't find the container with id ce633f4010f5a47363a9e999f232518cda87f5a33e05abc9afeb6aa6c0df9aca
Jul 15 10:55:20.992441 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:20.992408    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf" event={"ID":"c0f17d07-73e9-4bab-b471-977e8074167f","Type":"ContainerStarted","Data":"ce633f4010f5a47363a9e999f232518cda87f5a33e05abc9afeb6aa6c0df9aca"}
Jul 15 10:55:26.019017 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:26.018985    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf" event={"ID":"c0f17d07-73e9-4bab-b471-977e8074167f","Type":"ContainerStarted","Data":"bfc179249805aebddfc9216677e8d33b781868ee05b48c3e7487f4eab6cc3c2b"}
Jul 15 10:55:26.019428 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:26.019168    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"
Jul 15 10:55:26.048175 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:26.048130    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf" podStartSLOduration=2.447711786 podStartE2EDuration="7.048114896s" totalImagesPullingTime="4.60040311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:55:26.046983868 +0000 UTC m=+603.846457396" watchObservedRunningTime="2026-07-15 10:55:26.048114896 +0000 UTC m=+603.847588421"
Jul 15 10:55:27.476284 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.476254    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"]
Jul 15 10:55:27.479864 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.479848    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:27.484899 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.484880    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 15 10:55:27.485013 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.484946    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fl9gn\""
Jul 15 10:55:27.485537 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.485517    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 15 10:55:27.546652 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.546630    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"]
Jul 15 10:55:27.591193 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.591173    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/55a8418e-50db-4c57-b760-be2777bedc3a-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-jng46\" (UID: \"55a8418e-50db-4c57-b760-be2777bedc3a\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:27.591315 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.591231    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djvqc\" (UniqueName: \"kubernetes.io/projected/55a8418e-50db-4c57-b760-be2777bedc3a-kube-api-access-djvqc\") pod \"kuadrant-console-plugin-85fd4b8b9d-jng46\" (UID: \"55a8418e-50db-4c57-b760-be2777bedc3a\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:27.591366 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.591317    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/55a8418e-50db-4c57-b760-be2777bedc3a-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-jng46\" (UID: \"55a8418e-50db-4c57-b760-be2777bedc3a\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:27.692488 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.692461    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djvqc\" (UniqueName: \"kubernetes.io/projected/55a8418e-50db-4c57-b760-be2777bedc3a-kube-api-access-djvqc\") pod \"kuadrant-console-plugin-85fd4b8b9d-jng46\" (UID: \"55a8418e-50db-4c57-b760-be2777bedc3a\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:27.692581 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.692505    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/55a8418e-50db-4c57-b760-be2777bedc3a-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-jng46\" (UID: \"55a8418e-50db-4c57-b760-be2777bedc3a\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:27.692581 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.692559    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/55a8418e-50db-4c57-b760-be2777bedc3a-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-jng46\" (UID: \"55a8418e-50db-4c57-b760-be2777bedc3a\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:27.692731 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:55:27.692712    2536 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 15 10:55:27.692812 ip-10-0-134-178 kubenswrapper[2536]: E0715 10:55:27.692799    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a8418e-50db-4c57-b760-be2777bedc3a-plugin-serving-cert podName:55a8418e-50db-4c57-b760-be2777bedc3a nodeName:}" failed. No retries permitted until 2026-07-15 10:55:28.192775606 +0000 UTC m=+605.992249114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/55a8418e-50db-4c57-b760-be2777bedc3a-plugin-serving-cert") pod "kuadrant-console-plugin-85fd4b8b9d-jng46" (UID: "55a8418e-50db-4c57-b760-be2777bedc3a") : secret "plugin-serving-cert" not found
Jul 15 10:55:27.693113 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.693095    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/55a8418e-50db-4c57-b760-be2777bedc3a-nginx-conf\") pod \"kuadrant-console-plugin-85fd4b8b9d-jng46\" (UID: \"55a8418e-50db-4c57-b760-be2777bedc3a\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:27.710173 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:27.710144    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djvqc\" (UniqueName: \"kubernetes.io/projected/55a8418e-50db-4c57-b760-be2777bedc3a-kube-api-access-djvqc\") pod \"kuadrant-console-plugin-85fd4b8b9d-jng46\" (UID: \"55a8418e-50db-4c57-b760-be2777bedc3a\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:28.196699 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:28.196664    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/55a8418e-50db-4c57-b760-be2777bedc3a-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-jng46\" (UID: \"55a8418e-50db-4c57-b760-be2777bedc3a\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:28.199090 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:28.199060    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/55a8418e-50db-4c57-b760-be2777bedc3a-plugin-serving-cert\") pod \"kuadrant-console-plugin-85fd4b8b9d-jng46\" (UID: \"55a8418e-50db-4c57-b760-be2777bedc3a\") " pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:28.389408 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:28.389385    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"
Jul 15 10:55:28.530182 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:28.530157    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46"]
Jul 15 10:55:28.532766 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:55:28.532734    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55a8418e_50db_4c57_b760_be2777bedc3a.slice/crio-6c5187135f39e41ee9e6488e078c8bdaf57efcc6757e475e0047bc4fc91f78f6 WatchSource:0}: Error finding container 6c5187135f39e41ee9e6488e078c8bdaf57efcc6757e475e0047bc4fc91f78f6: Status 404 returned error can't find the container with id 6c5187135f39e41ee9e6488e078c8bdaf57efcc6757e475e0047bc4fc91f78f6
Jul 15 10:55:29.034112 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:29.034077    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46" event={"ID":"55a8418e-50db-4c57-b760-be2777bedc3a","Type":"ContainerStarted","Data":"6c5187135f39e41ee9e6488e078c8bdaf57efcc6757e475e0047bc4fc91f78f6"}
Jul 15 10:55:33.055539 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:33.055512    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46" event={"ID":"55a8418e-50db-4c57-b760-be2777bedc3a","Type":"ContainerStarted","Data":"d582e174867bb1109b9bc4be2bd0d67ddd1d337ca04e2f555274a1c9adb24ed7"}
Jul 15 10:55:33.079535 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:33.079482    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-jng46" podStartSLOduration=1.632205945 podStartE2EDuration="6.079466179s" totalImagesPullingTime="4.447260234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:55:33.078649885 +0000 UTC m=+610.878123415" watchObservedRunningTime="2026-07-15 10:55:33.079466179 +0000 UTC m=+610.878939703"
Jul 15 10:55:37.025543 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:55:37.025509    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-frhbf"
Jul 15 10:56:10.919969 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:10.919937    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-p5c6m"]
Jul 15 10:56:10.923202 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:10.923185    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m"
Jul 15 10:56:10.925662 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:10.925641    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 15 10:56:10.931957 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:10.931935    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-p5c6m"]
Jul 15 10:56:10.971487 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:10.971464    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-p5c6m"]
Jul 15 10:56:11.038054 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:11.038021    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee-config-file\") pod \"limitador-limitador-69574b596d-p5c6m\" (UID: \"664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee\") " pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m"
Jul 15 10:56:11.038170 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:11.038068    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fclj\" (UniqueName: \"kubernetes.io/projected/664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee-kube-api-access-2fclj\") pod \"limitador-limitador-69574b596d-p5c6m\" (UID: \"664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee\") " pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m"
Jul 15 10:56:11.139360 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:11.139338    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee-config-file\") pod \"limitador-limitador-69574b596d-p5c6m\" (UID: \"664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee\") " pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m"
Jul 15 10:56:11.139477 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:11.139382    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fclj\" (UniqueName: \"kubernetes.io/projected/664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee-kube-api-access-2fclj\") pod \"limitador-limitador-69574b596d-p5c6m\" (UID: \"664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee\") " pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m"
Jul 15 10:56:11.139946 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:11.139929    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee-config-file\") pod \"limitador-limitador-69574b596d-p5c6m\" (UID: \"664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee\") " pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m"
Jul 15 10:56:11.147833 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:11.147809    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fclj\" (UniqueName: \"kubernetes.io/projected/664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee-kube-api-access-2fclj\") pod \"limitador-limitador-69574b596d-p5c6m\" (UID: \"664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee\") " pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m"
Jul 15 10:56:11.233993 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:11.233963    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m"
Jul 15 10:56:11.365064 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:11.365039    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-p5c6m"]
Jul 15 10:56:11.367121 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:56:11.367095    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod664e3f9c_a7b3_4eeb_8c35_a72a9a5410ee.slice/crio-ceb8331096b97d855bb9083c2e6c5f90a9834b77fb0ea66cb9d6780b1b1c6abb WatchSource:0}: Error finding container ceb8331096b97d855bb9083c2e6c5f90a9834b77fb0ea66cb9d6780b1b1c6abb: Status 404 returned error can't find the container with id ceb8331096b97d855bb9083c2e6c5f90a9834b77fb0ea66cb9d6780b1b1c6abb
Jul 15 10:56:12.235138 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:12.235101    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m" event={"ID":"664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee","Type":"ContainerStarted","Data":"ceb8331096b97d855bb9083c2e6c5f90a9834b77fb0ea66cb9d6780b1b1c6abb"}
Jul 15 10:56:15.251531 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:15.251494    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m" event={"ID":"664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee","Type":"ContainerStarted","Data":"16869adc95ac1bfcd6ea6788aad267f10c672521a6416bf9c4256a8875cf018e"}
Jul 15 10:56:15.251930 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:15.251619    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m"
Jul 15 10:56:15.269856 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:15.269813    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m" podStartSLOduration=1.714520619 podStartE2EDuration="5.269793907s" totalImagesPullingTime="3.555273288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:56:15.267782982 +0000 UTC m=+653.067256499" watchObservedRunningTime="2026-07-15 10:56:15.269793907 +0000 UTC m=+653.069267432"
Jul 15 10:56:26.257487 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:56:26.257456    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-69574b596d-p5c6m"
Jul 15 10:59:46.917688 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.917658    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"]
Jul 15 10:59:46.920962 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.920942    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:46.924470 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.924440    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 10:59:46.925097 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.925079    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 10:59:46.925183 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.925138    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs\""
Jul 15 10:59:46.925323 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.925145    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 10:59:46.935376 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.935357    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"]
Jul 15 10:59:46.994083 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.994060    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:46.994171 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.994095    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d21993b3-0e9e-4a36-a599-0d298d42dda6-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:46.994171 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.994117    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jv94\" (UniqueName: \"kubernetes.io/projected/d21993b3-0e9e-4a36-a599-0d298d42dda6-kube-api-access-6jv94\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:46.994171 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.994160    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:46.994304 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.994191    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:46.994304 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:46.994247    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.095178 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.095150    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.095267 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.095194    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.095267 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.095230    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.095403 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.095264    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d21993b3-0e9e-4a36-a599-0d298d42dda6-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.095403 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.095320    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jv94\" (UniqueName: \"kubernetes.io/projected/d21993b3-0e9e-4a36-a599-0d298d42dda6-kube-api-access-6jv94\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.095403 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.095359    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.095617 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.095594    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.095617 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.095610    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.095750 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.095733    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.097640 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.097620    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.097791 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.097773    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d21993b3-0e9e-4a36-a599-0d298d42dda6-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.105259 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.105231    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jv94\" (UniqueName: \"kubernetes.io/projected/d21993b3-0e9e-4a36-a599-0d298d42dda6-kube-api-access-6jv94\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") " pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.231420 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.231395    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:47.388912 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.388884    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"]
Jul 15 10:59:47.391355 ip-10-0-134-178 kubenswrapper[2536]: W0715 10:59:47.391327    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21993b3_0e9e_4a36_a599_0d298d42dda6.slice/crio-2ed669f891bb3c18757f80cd5f52e4f345603479356a608e5e9e433924a1e3a0 WatchSource:0}: Error finding container 2ed669f891bb3c18757f80cd5f52e4f345603479356a608e5e9e433924a1e3a0: Status 404 returned error can't find the container with id 2ed669f891bb3c18757f80cd5f52e4f345603479356a608e5e9e433924a1e3a0
Jul 15 10:59:47.393478 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:47.393459    2536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 10:59:48.187525 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:48.187488    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd" event={"ID":"d21993b3-0e9e-4a36-a599-0d298d42dda6","Type":"ContainerStarted","Data":"2ed669f891bb3c18757f80cd5f52e4f345603479356a608e5e9e433924a1e3a0"}
Jul 15 10:59:49.193997 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:49.193964    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd" event={"ID":"d21993b3-0e9e-4a36-a599-0d298d42dda6","Type":"ContainerStarted","Data":"0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d"}
Jul 15 10:59:49.231985 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:49.231956    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:57.231676 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:57.231647    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:57.244035 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:57.244006    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 10:59:57.263926 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:57.263878    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd" podStartSLOduration=10.237610178 podStartE2EDuration="11.263864112s" totalImagesPullingTime="1.026253934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 10:59:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 10:59:49.231007872 +0000 UTC m=+867.030481401" watchObservedRunningTime="2026-07-15 10:59:57.263864112 +0000 UTC m=+875.063337637"
Jul 15 10:59:58.242868 ip-10-0-134-178 kubenswrapper[2536]: I0715 10:59:58.242844    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 11:00:00.234250 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:00.234219    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"]
Jul 15 11:00:15.265097 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.265033    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd" podUID="d21993b3-0e9e-4a36-a599-0d298d42dda6" containerName="main" containerID="cri-o://0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d" gracePeriod=45
Jul 15 11:00:15.523209 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.523152    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 11:00:15.613541 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613514    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-home\") pod \"d21993b3-0e9e-4a36-a599-0d298d42dda6\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") "
Jul 15 11:00:15.613664 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613571    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jv94\" (UniqueName: \"kubernetes.io/projected/d21993b3-0e9e-4a36-a599-0d298d42dda6-kube-api-access-6jv94\") pod \"d21993b3-0e9e-4a36-a599-0d298d42dda6\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") "
Jul 15 11:00:15.613664 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613599    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-dshm\") pod \"d21993b3-0e9e-4a36-a599-0d298d42dda6\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") "
Jul 15 11:00:15.613664 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613626    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-tmp-dir\") pod \"d21993b3-0e9e-4a36-a599-0d298d42dda6\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") "
Jul 15 11:00:15.613664 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613642    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-model-cache\") pod \"d21993b3-0e9e-4a36-a599-0d298d42dda6\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") "
Jul 15 11:00:15.613867 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613679    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d21993b3-0e9e-4a36-a599-0d298d42dda6-tls-certs\") pod \"d21993b3-0e9e-4a36-a599-0d298d42dda6\" (UID: \"d21993b3-0e9e-4a36-a599-0d298d42dda6\") "
Jul 15 11:00:15.613867 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613778    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-home" (OuterVolumeSpecName: "home") pod "d21993b3-0e9e-4a36-a599-0d298d42dda6" (UID: "d21993b3-0e9e-4a36-a599-0d298d42dda6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:00:15.613958 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613873    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d21993b3-0e9e-4a36-a599-0d298d42dda6" (UID: "d21993b3-0e9e-4a36-a599-0d298d42dda6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:00:15.613958 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613897    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-model-cache" (OuterVolumeSpecName: "model-cache") pod "d21993b3-0e9e-4a36-a599-0d298d42dda6" (UID: "d21993b3-0e9e-4a36-a599-0d298d42dda6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:00:15.614040 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613964    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:00:15.614040 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613981    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:00:15.614040 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.613996    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:00:15.615826 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.615798    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-dshm" (OuterVolumeSpecName: "dshm") pod "d21993b3-0e9e-4a36-a599-0d298d42dda6" (UID: "d21993b3-0e9e-4a36-a599-0d298d42dda6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:00:15.615930 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.615896    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21993b3-0e9e-4a36-a599-0d298d42dda6-kube-api-access-6jv94" (OuterVolumeSpecName: "kube-api-access-6jv94") pod "d21993b3-0e9e-4a36-a599-0d298d42dda6" (UID: "d21993b3-0e9e-4a36-a599-0d298d42dda6"). InnerVolumeSpecName "kube-api-access-6jv94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:00:15.616031 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.616006    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d21993b3-0e9e-4a36-a599-0d298d42dda6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d21993b3-0e9e-4a36-a599-0d298d42dda6" (UID: "d21993b3-0e9e-4a36-a599-0d298d42dda6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:00:15.714543 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.714515    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jv94\" (UniqueName: \"kubernetes.io/projected/d21993b3-0e9e-4a36-a599-0d298d42dda6-kube-api-access-6jv94\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:00:15.714543 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.714538    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d21993b3-0e9e-4a36-a599-0d298d42dda6-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:00:15.714704 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:15.714551    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d21993b3-0e9e-4a36-a599-0d298d42dda6-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:00:16.311772 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:16.311742    2536 generic.go:356] "Generic (PLEG): container finished" podID="d21993b3-0e9e-4a36-a599-0d298d42dda6" containerID="0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d" exitCode=0
Jul 15 11:00:16.312171 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:16.311789    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd" event={"ID":"d21993b3-0e9e-4a36-a599-0d298d42dda6","Type":"ContainerDied","Data":"0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d"}
Jul 15 11:00:16.312171 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:16.311800    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"
Jul 15 11:00:16.312171 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:16.311831    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd" event={"ID":"d21993b3-0e9e-4a36-a599-0d298d42dda6","Type":"ContainerDied","Data":"2ed669f891bb3c18757f80cd5f52e4f345603479356a608e5e9e433924a1e3a0"}
Jul 15 11:00:16.312171 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:16.311853    2536 scope.go:117] "RemoveContainer" containerID="0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d"
Jul 15 11:00:16.320782 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:16.320763    2536 scope.go:117] "RemoveContainer" containerID="0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d"
Jul 15 11:00:16.321050 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:00:16.321033    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d\": container with ID starting with 0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d not found: ID does not exist" containerID="0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d"
Jul 15 11:00:16.321095 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:16.321059    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d"} err="failed to get container status \"0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d\": rpc error: code = NotFound desc = could not find container \"0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d\": container with ID starting with 0b0cb5f69c1af6d3420f59036a63e0b77b51044b86b82f1e222aa24d37e6b09d not found: ID does not exist"
Jul 15 11:00:16.333046 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:16.333021    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"]
Jul 15 11:00:16.337316 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:16.337269    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5bwrcdd"]
Jul 15 11:00:16.594581 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:00:16.594518    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d21993b3-0e9e-4a36-a599-0d298d42dda6" path="/var/lib/kubelet/pods/d21993b3-0e9e-4a36-a599-0d298d42dda6/volumes"
Jul 15 11:01:51.314383 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.314343    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"]
Jul 15 11:01:51.314965 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.314722    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21993b3-0e9e-4a36-a599-0d298d42dda6" containerName="main"
Jul 15 11:01:51.314965 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.314734    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21993b3-0e9e-4a36-a599-0d298d42dda6" containerName="main"
Jul 15 11:01:51.314965 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.314820    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21993b3-0e9e-4a36-a599-0d298d42dda6" containerName="main"
Jul 15 11:01:51.317817 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.317796    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.321611 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.321587    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:01:51.321732 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.321639    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:01:51.321732 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.321586    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs\""
Jul 15 11:01:51.321732 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.321684    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:01:51.329596 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.329572    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"]
Jul 15 11:01:51.361679 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.361655    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.361780 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.361688    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.361780 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.361713    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.361850 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.361784    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.361850 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.361815    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.361850 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.361844    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsjqp\" (UniqueName: \"kubernetes.io/projected/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-kube-api-access-lsjqp\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.462709 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.462676    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.462841 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.462721    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsjqp\" (UniqueName: \"kubernetes.io/projected/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-kube-api-access-lsjqp\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.462841 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.462753    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.462841 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.462777    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.462841 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.462803    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.462841 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.462833    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.463147 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.463127    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.463207 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.463154    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.463207 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.463134    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.465174 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.465148    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.465502 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.465481    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.471819 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.471799    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsjqp\" (UniqueName: \"kubernetes.io/projected/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-kube-api-access-lsjqp\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.628778 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.628721    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:01:51.772413 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:51.772364    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"]
Jul 15 11:01:51.774816 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:01:51.774786    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02abbfa7_1c52_48d2_82f0_f2d83ff823e9.slice/crio-b5217c21c2fe203e13ec760622df6ada3e4de8ccbcd89c6a75a4d447187d1eb5 WatchSource:0}: Error finding container b5217c21c2fe203e13ec760622df6ada3e4de8ccbcd89c6a75a4d447187d1eb5: Status 404 returned error can't find the container with id b5217c21c2fe203e13ec760622df6ada3e4de8ccbcd89c6a75a4d447187d1eb5
Jul 15 11:01:52.733197 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:52.733162    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f" event={"ID":"02abbfa7-1c52-48d2-82f0-f2d83ff823e9","Type":"ContainerStarted","Data":"1556c72084387e9c14d27fa5a9b2da27f0533c7300ae210efe2e5ac6af3d2f34"}
Jul 15 11:01:52.733684 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:52.733212    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f" event={"ID":"02abbfa7-1c52-48d2-82f0-f2d83ff823e9","Type":"ContainerStarted","Data":"b5217c21c2fe203e13ec760622df6ada3e4de8ccbcd89c6a75a4d447187d1eb5"}
Jul 15 11:01:52.751416 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:52.751367    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f" podStartSLOduration=1.7513477800000001 podStartE2EDuration="1.75134778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:01:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:01:52.750262709 +0000 UTC m=+990.549736234" watchObservedRunningTime="2026-07-15 11:01:52.75134778 +0000 UTC m=+990.550821304"
Jul 15 11:01:53.629775 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:01:53.629747    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:02:01.629646 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:01.629610    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:02:01.641954 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:01.641931    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:02:01.781735 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:01.781704    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:02:24.790448 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:24.790418    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"]
Jul 15 11:02:39.810911 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:39.810847    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f" podUID="02abbfa7-1c52-48d2-82f0-f2d83ff823e9" containerName="main" containerID="cri-o://1556c72084387e9c14d27fa5a9b2da27f0533c7300ae210efe2e5ac6af3d2f34" gracePeriod=45
Jul 15 11:02:39.938462 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:39.938432    2536 generic.go:356] "Generic (PLEG): container finished" podID="02abbfa7-1c52-48d2-82f0-f2d83ff823e9" containerID="1556c72084387e9c14d27fa5a9b2da27f0533c7300ae210efe2e5ac6af3d2f34" exitCode=0
Jul 15 11:02:39.938580 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:39.938510    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f" event={"ID":"02abbfa7-1c52-48d2-82f0-f2d83ff823e9","Type":"ContainerDied","Data":"1556c72084387e9c14d27fa5a9b2da27f0533c7300ae210efe2e5ac6af3d2f34"}
Jul 15 11:02:40.061409 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.061342    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:02:40.069999 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.069977    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-home\") pod \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") "
Jul 15 11:02:40.070130 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.070031    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-dshm\") pod \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") "
Jul 15 11:02:40.070130 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.070062    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-model-cache\") pod \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") "
Jul 15 11:02:40.070130 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.070079    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsjqp\" (UniqueName: \"kubernetes.io/projected/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-kube-api-access-lsjqp\") pod \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") "
Jul 15 11:02:40.070130 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.070121    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tmp-dir\") pod \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") "
Jul 15 11:02:40.070372 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.070157    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tls-certs\") pod \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\" (UID: \"02abbfa7-1c52-48d2-82f0-f2d83ff823e9\") "
Jul 15 11:02:40.070372 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.070206    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-home" (OuterVolumeSpecName: "home") pod "02abbfa7-1c52-48d2-82f0-f2d83ff823e9" (UID: "02abbfa7-1c52-48d2-82f0-f2d83ff823e9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:02:40.070498 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.070405    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "02abbfa7-1c52-48d2-82f0-f2d83ff823e9" (UID: "02abbfa7-1c52-48d2-82f0-f2d83ff823e9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:02:40.070498 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.070417    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-model-cache" (OuterVolumeSpecName: "model-cache") pod "02abbfa7-1c52-48d2-82f0-f2d83ff823e9" (UID: "02abbfa7-1c52-48d2-82f0-f2d83ff823e9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:02:40.070594 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.070496    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:02:40.070594 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.070515    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:02:40.070594 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.070528    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:02:40.072259 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.072233    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-dshm" (OuterVolumeSpecName: "dshm") pod "02abbfa7-1c52-48d2-82f0-f2d83ff823e9" (UID: "02abbfa7-1c52-48d2-82f0-f2d83ff823e9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:02:40.072405 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.072251    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-kube-api-access-lsjqp" (OuterVolumeSpecName: "kube-api-access-lsjqp") pod "02abbfa7-1c52-48d2-82f0-f2d83ff823e9" (UID: "02abbfa7-1c52-48d2-82f0-f2d83ff823e9"). InnerVolumeSpecName "kube-api-access-lsjqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:02:40.072449 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.072395    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "02abbfa7-1c52-48d2-82f0-f2d83ff823e9" (UID: "02abbfa7-1c52-48d2-82f0-f2d83ff823e9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:02:40.171100 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.171073    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:02:40.171100 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.171099    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsjqp\" (UniqueName: \"kubernetes.io/projected/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-kube-api-access-lsjqp\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:02:40.171256 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.171111    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02abbfa7-1c52-48d2-82f0-f2d83ff823e9-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:02:40.944500 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.944421    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"
Jul 15 11:02:40.944500 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.944450    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f" event={"ID":"02abbfa7-1c52-48d2-82f0-f2d83ff823e9","Type":"ContainerDied","Data":"b5217c21c2fe203e13ec760622df6ada3e4de8ccbcd89c6a75a4d447187d1eb5"}
Jul 15 11:02:40.944992 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.944505    2536 scope.go:117] "RemoveContainer" containerID="1556c72084387e9c14d27fa5a9b2da27f0533c7300ae210efe2e5ac6af3d2f34"
Jul 15 11:02:40.960615 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.960591    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"]
Jul 15 11:02:40.964107 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:40.964085    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879rgq2f"]
Jul 15 11:02:42.594499 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:42.594461    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02abbfa7-1c52-48d2-82f0-f2d83ff823e9" path="/var/lib/kubelet/pods/02abbfa7-1c52-48d2-82f0-f2d83ff823e9/volumes"
Jul 15 11:02:47.921738 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:47.921707    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"]
Jul 15 11:02:47.922116 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:47.922092    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02abbfa7-1c52-48d2-82f0-f2d83ff823e9" containerName="main"
Jul 15 11:02:47.922116 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:47.922106    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="02abbfa7-1c52-48d2-82f0-f2d83ff823e9" containerName="main"
Jul 15 11:02:47.922203 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:47.922160    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="02abbfa7-1c52-48d2-82f0-f2d83ff823e9" containerName="main"
Jul 15 11:02:47.924875 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:47.924856    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:47.928656 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:47.928636    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:02:47.929564 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:47.929540    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs\""
Jul 15 11:02:47.929564 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:47.929548    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:02:47.929704 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:47.929598    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:02:47.936179 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:47.936156    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"]
Jul 15 11:02:48.035696 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.035664    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxwm9\" (UniqueName: \"kubernetes.io/projected/52dbd373-8a10-4156-8669-d6c7a2afb9d2-kube-api-access-kxwm9\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.035821 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.035713    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.035821 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.035809    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.035900 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.035834    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.035900 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.035886    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.035962 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.035920    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.136778 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.136752    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.136888 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.136790    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.136888 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.136833    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxwm9\" (UniqueName: \"kubernetes.io/projected/52dbd373-8a10-4156-8669-d6c7a2afb9d2-kube-api-access-kxwm9\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.136888 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.136866    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.137009 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.136899    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.137009 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.136916    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.137336 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.137317    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.137424 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.137354    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.137471 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.137417    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.139233 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.139213    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.139342 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.139325    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.145458 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.145436    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxwm9\" (UniqueName: \"kubernetes.io/projected/52dbd373-8a10-4156-8669-d6c7a2afb9d2-kube-api-access-kxwm9\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.235529 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.235505    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:48.375692 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.375668    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"]
Jul 15 11:02:48.380298 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:02:48.377898    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52dbd373_8a10_4156_8669_d6c7a2afb9d2.slice/crio-b7c750a3207c61181fef3b9f2617be4b88c732faca85d375bea357c5ab3955d2 WatchSource:0}: Error finding container b7c750a3207c61181fef3b9f2617be4b88c732faca85d375bea357c5ab3955d2: Status 404 returned error can't find the container with id b7c750a3207c61181fef3b9f2617be4b88c732faca85d375bea357c5ab3955d2
Jul 15 11:02:48.979914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.979875    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb" event={"ID":"52dbd373-8a10-4156-8669-d6c7a2afb9d2","Type":"ContainerStarted","Data":"578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838"}
Jul 15 11:02:48.979914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.979923    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb" event={"ID":"52dbd373-8a10-4156-8669-d6c7a2afb9d2","Type":"ContainerStarted","Data":"b7c750a3207c61181fef3b9f2617be4b88c732faca85d375bea357c5ab3955d2"}
Jul 15 11:02:48.999605 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:48.999555    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb" podStartSLOduration=1.999540021 podStartE2EDuration="1.999540021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:02:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:02:48.998468539 +0000 UTC m=+1046.797942065" watchObservedRunningTime="2026-07-15 11:02:48.999540021 +0000 UTC m=+1046.799013545"
Jul 15 11:02:49.236731 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:49.236649    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:58.236627 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:58.236576    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:58.249029 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:58.249003    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:02:59.037926 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:02:59.037897    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:03:57.905595 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:57.905554    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"]
Jul 15 11:03:57.909220 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:57.909203    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:57.911534 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:57.911513    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Jul 15 11:03:57.919940 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:57.919918    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"]
Jul 15 11:03:58.027676 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.027638    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.027833 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.027687    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.027833 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.027712    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqhhn\" (UniqueName: \"kubernetes.io/projected/c00a8fed-ea41-4b6d-a22b-efa63320929d-kube-api-access-kqhhn\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.027833 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.027759    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.027833 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.027816    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.027968 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.027845    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00a8fed-ea41-4b6d-a22b-efa63320929d-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.129265 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.129233    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.129433 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.129294    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.129433 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.129323    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqhhn\" (UniqueName: \"kubernetes.io/projected/c00a8fed-ea41-4b6d-a22b-efa63320929d-kube-api-access-kqhhn\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.129433 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.129348    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.129433 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.129388    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.129433 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.129423    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00a8fed-ea41-4b6d-a22b-efa63320929d-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.129767 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.129746    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.129830 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.129811    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.129924 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.129830    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.131735 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.131714    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.131848 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.131831    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00a8fed-ea41-4b6d-a22b-efa63320929d-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.138030 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.138008    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqhhn\" (UniqueName: \"kubernetes.io/projected/c00a8fed-ea41-4b6d-a22b-efa63320929d-kube-api-access-kqhhn\") pod \"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.219830 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.219796    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:03:58.357789 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:58.357765    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"]
Jul 15 11:03:58.360447 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:03:58.360416    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00a8fed_ea41_4b6d_a22b_efa63320929d.slice/crio-ce56b207387892cf46b6b1bebdff4a4b429b01079e6d49a85aae4d1ebf46638b WatchSource:0}: Error finding container ce56b207387892cf46b6b1bebdff4a4b429b01079e6d49a85aae4d1ebf46638b: Status 404 returned error can't find the container with id ce56b207387892cf46b6b1bebdff4a4b429b01079e6d49a85aae4d1ebf46638b
Jul 15 11:03:59.301671 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:59.301631    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r" event={"ID":"c00a8fed-ea41-4b6d-a22b-efa63320929d","Type":"ContainerStarted","Data":"362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead"}
Jul 15 11:03:59.302133 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:59.301684    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r" event={"ID":"c00a8fed-ea41-4b6d-a22b-efa63320929d","Type":"ContainerStarted","Data":"ce56b207387892cf46b6b1bebdff4a4b429b01079e6d49a85aae4d1ebf46638b"}
Jul 15 11:03:59.326646 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:03:59.326596    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r" podStartSLOduration=2.326580596 podStartE2EDuration="2.326580596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:03:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:03:59.324486138 +0000 UTC m=+1117.123959664" watchObservedRunningTime="2026-07-15 11:03:59.326580596 +0000 UTC m=+1117.126054123"
Jul 15 11:04:00.220138 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:00.220106    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:04:08.220316 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:08.220257    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:04:08.232915 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:08.232886    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:04:08.358317 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:08.358284    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:04:31.394836 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:31.394803    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"]
Jul 15 11:04:46.415609 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.415502    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r" podUID="c00a8fed-ea41-4b6d-a22b-efa63320929d" containerName="main" containerID="cri-o://362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead" gracePeriod=45
Jul 15 11:04:46.664505 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.664474    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:04:46.755874 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.755844    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-dshm\") pod \"c00a8fed-ea41-4b6d-a22b-efa63320929d\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") "
Jul 15 11:04:46.756032 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.755877    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00a8fed-ea41-4b6d-a22b-efa63320929d-tls-certs\") pod \"c00a8fed-ea41-4b6d-a22b-efa63320929d\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") "
Jul 15 11:04:46.756032 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.755913    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-tmp-dir\") pod \"c00a8fed-ea41-4b6d-a22b-efa63320929d\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") "
Jul 15 11:04:46.756032 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.755928    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-home\") pod \"c00a8fed-ea41-4b6d-a22b-efa63320929d\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") "
Jul 15 11:04:46.756032 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.755978    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqhhn\" (UniqueName: \"kubernetes.io/projected/c00a8fed-ea41-4b6d-a22b-efa63320929d-kube-api-access-kqhhn\") pod \"c00a8fed-ea41-4b6d-a22b-efa63320929d\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") "
Jul 15 11:04:46.756032 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.756025    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-model-cache\") pod \"c00a8fed-ea41-4b6d-a22b-efa63320929d\" (UID: \"c00a8fed-ea41-4b6d-a22b-efa63320929d\") "
Jul 15 11:04:46.756496 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.756174    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c00a8fed-ea41-4b6d-a22b-efa63320929d" (UID: "c00a8fed-ea41-4b6d-a22b-efa63320929d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:04:46.756496 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.756228    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-home" (OuterVolumeSpecName: "home") pod "c00a8fed-ea41-4b6d-a22b-efa63320929d" (UID: "c00a8fed-ea41-4b6d-a22b-efa63320929d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:04:46.756496 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.756354    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:04:46.756496 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.756374    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:04:46.756496 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.756374    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-model-cache" (OuterVolumeSpecName: "model-cache") pod "c00a8fed-ea41-4b6d-a22b-efa63320929d" (UID: "c00a8fed-ea41-4b6d-a22b-efa63320929d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:04:46.758159 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.758139    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-dshm" (OuterVolumeSpecName: "dshm") pod "c00a8fed-ea41-4b6d-a22b-efa63320929d" (UID: "c00a8fed-ea41-4b6d-a22b-efa63320929d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:04:46.758245 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.758227    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c00a8fed-ea41-4b6d-a22b-efa63320929d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c00a8fed-ea41-4b6d-a22b-efa63320929d" (UID: "c00a8fed-ea41-4b6d-a22b-efa63320929d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:04:46.758334 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.758315    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c00a8fed-ea41-4b6d-a22b-efa63320929d-kube-api-access-kqhhn" (OuterVolumeSpecName: "kube-api-access-kqhhn") pod "c00a8fed-ea41-4b6d-a22b-efa63320929d" (UID: "c00a8fed-ea41-4b6d-a22b-efa63320929d"). InnerVolumeSpecName "kube-api-access-kqhhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:04:46.856784 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.856754    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqhhn\" (UniqueName: \"kubernetes.io/projected/c00a8fed-ea41-4b6d-a22b-efa63320929d-kube-api-access-kqhhn\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:04:46.856784 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.856780    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:04:46.856922 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.856789    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00a8fed-ea41-4b6d-a22b-efa63320929d-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:04:46.856922 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:46.856797    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00a8fed-ea41-4b6d-a22b-efa63320929d-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:04:47.533645 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:47.533605    2536 generic.go:356] "Generic (PLEG): container finished" podID="c00a8fed-ea41-4b6d-a22b-efa63320929d" containerID="362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead" exitCode=0
Jul 15 11:04:47.533645 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:47.533646    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"
Jul 15 11:04:47.534126 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:47.533682    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r" event={"ID":"c00a8fed-ea41-4b6d-a22b-efa63320929d","Type":"ContainerDied","Data":"362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead"}
Jul 15 11:04:47.534126 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:47.533741    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r" event={"ID":"c00a8fed-ea41-4b6d-a22b-efa63320929d","Type":"ContainerDied","Data":"ce56b207387892cf46b6b1bebdff4a4b429b01079e6d49a85aae4d1ebf46638b"}
Jul 15 11:04:47.534126 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:47.533764    2536 scope.go:117] "RemoveContainer" containerID="362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead"
Jul 15 11:04:47.543109 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:47.543093    2536 scope.go:117] "RemoveContainer" containerID="362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead"
Jul 15 11:04:47.543390 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:04:47.543372    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead\": container with ID starting with 362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead not found: ID does not exist" containerID="362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead"
Jul 15 11:04:47.543452 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:47.543398    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead"} err="failed to get container status \"362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead\": rpc error: code = NotFound desc = could not find container \"362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead\": container with ID starting with 362f202fd78d487cc09c0192551bc326654d12124ab42452f78e79f2454d8ead not found: ID does not exist"
Jul 15 11:04:47.555559 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:47.555531    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"]
Jul 15 11:04:47.560424 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:47.560364    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58ddqj45r"]
Jul 15 11:04:48.594732 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:48.594694    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c00a8fed-ea41-4b6d-a22b-efa63320929d" path="/var/lib/kubelet/pods/c00a8fed-ea41-4b6d-a22b-efa63320929d/volumes"
Jul 15 11:04:51.859746 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:04:51.859712    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"]
Jul 15 11:05:06.880321 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:06.880234    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb" podUID="52dbd373-8a10-4156-8669-d6c7a2afb9d2" containerName="main" containerID="cri-o://578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838" gracePeriod=45
Jul 15 11:05:07.027420 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.027351    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb" podUID="52dbd373-8a10-4156-8669-d6c7a2afb9d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.33:8000/health\": dial tcp 10.132.0.33:8000: connect: connection refused"
Jul 15 11:05:07.135009 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.134935    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:05:07.227964 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.227926    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tmp-dir\") pod \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") "
Jul 15 11:05:07.227964 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.227971    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tls-certs\") pod \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") "
Jul 15 11:05:07.228223 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.228024    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-dshm\") pod \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") "
Jul 15 11:05:07.228223 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.228039    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-home\") pod \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") "
Jul 15 11:05:07.228223 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.228060    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxwm9\" (UniqueName: \"kubernetes.io/projected/52dbd373-8a10-4156-8669-d6c7a2afb9d2-kube-api-access-kxwm9\") pod \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") "
Jul 15 11:05:07.228223 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.228134    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-model-cache\") pod \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\" (UID: \"52dbd373-8a10-4156-8669-d6c7a2afb9d2\") "
Jul 15 11:05:07.228223 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.228167    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "52dbd373-8a10-4156-8669-d6c7a2afb9d2" (UID: "52dbd373-8a10-4156-8669-d6c7a2afb9d2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:05:07.228487 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.228320    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-home" (OuterVolumeSpecName: "home") pod "52dbd373-8a10-4156-8669-d6c7a2afb9d2" (UID: "52dbd373-8a10-4156-8669-d6c7a2afb9d2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:05:07.228487 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.228431    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:05:07.228487 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.228447    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:05:07.228906 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.228458    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-model-cache" (OuterVolumeSpecName: "model-cache") pod "52dbd373-8a10-4156-8669-d6c7a2afb9d2" (UID: "52dbd373-8a10-4156-8669-d6c7a2afb9d2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:05:07.234951 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.231346    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-dshm" (OuterVolumeSpecName: "dshm") pod "52dbd373-8a10-4156-8669-d6c7a2afb9d2" (UID: "52dbd373-8a10-4156-8669-d6c7a2afb9d2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:05:07.236188 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.236155    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52dbd373-8a10-4156-8669-d6c7a2afb9d2" (UID: "52dbd373-8a10-4156-8669-d6c7a2afb9d2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:05:07.236625 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.236603    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52dbd373-8a10-4156-8669-d6c7a2afb9d2-kube-api-access-kxwm9" (OuterVolumeSpecName: "kube-api-access-kxwm9") pod "52dbd373-8a10-4156-8669-d6c7a2afb9d2" (UID: "52dbd373-8a10-4156-8669-d6c7a2afb9d2"). InnerVolumeSpecName "kube-api-access-kxwm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:05:07.329486 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.329456    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:05:07.329486 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.329481    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxwm9\" (UniqueName: \"kubernetes.io/projected/52dbd373-8a10-4156-8669-d6c7a2afb9d2-kube-api-access-kxwm9\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:05:07.329486 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.329494    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52dbd373-8a10-4156-8669-d6c7a2afb9d2-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:05:07.329724 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.329503    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52dbd373-8a10-4156-8669-d6c7a2afb9d2-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:05:07.628612 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.628577    2536 generic.go:356] "Generic (PLEG): container finished" podID="52dbd373-8a10-4156-8669-d6c7a2afb9d2" containerID="578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838" exitCode=0
Jul 15 11:05:07.628788 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.628641    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"
Jul 15 11:05:07.628788 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.628657    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb" event={"ID":"52dbd373-8a10-4156-8669-d6c7a2afb9d2","Type":"ContainerDied","Data":"578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838"}
Jul 15 11:05:07.628788 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.628722    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb" event={"ID":"52dbd373-8a10-4156-8669-d6c7a2afb9d2","Type":"ContainerDied","Data":"b7c750a3207c61181fef3b9f2617be4b88c732faca85d375bea357c5ab3955d2"}
Jul 15 11:05:07.628788 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.628750    2536 scope.go:117] "RemoveContainer" containerID="578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838"
Jul 15 11:05:07.638458 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.638440    2536 scope.go:117] "RemoveContainer" containerID="578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838"
Jul 15 11:05:07.638720 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:05:07.638702    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838\": container with ID starting with 578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838 not found: ID does not exist" containerID="578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838"
Jul 15 11:05:07.638763 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.638728    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838"} err="failed to get container status \"578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838\": rpc error: code = NotFound desc = could not find container \"578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838\": container with ID starting with 578c58652cb410fc0b7ecaf20c20401dd949ecdb407a9442d3121593ea5a4838 not found: ID does not exist"
Jul 15 11:05:07.652744 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.652717    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"]
Jul 15 11:05:07.655009 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:07.654986    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4b67bb"]
Jul 15 11:05:08.595795 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:08.595760    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52dbd373-8a10-4156-8669-d6c7a2afb9d2" path="/var/lib/kubelet/pods/52dbd373-8a10-4156-8669-d6c7a2afb9d2/volumes"
Jul 15 11:05:15.615465 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.615425    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"]
Jul 15 11:05:15.615914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.615792    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52dbd373-8a10-4156-8669-d6c7a2afb9d2" containerName="main"
Jul 15 11:05:15.615914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.615803    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="52dbd373-8a10-4156-8669-d6c7a2afb9d2" containerName="main"
Jul 15 11:05:15.615914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.615816    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c00a8fed-ea41-4b6d-a22b-efa63320929d" containerName="main"
Jul 15 11:05:15.615914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.615821    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="c00a8fed-ea41-4b6d-a22b-efa63320929d" containerName="main"
Jul 15 11:05:15.615914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.615883    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="52dbd373-8a10-4156-8669-d6c7a2afb9d2" containerName="main"
Jul 15 11:05:15.615914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.615893    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="c00a8fed-ea41-4b6d-a22b-efa63320929d" containerName="main"
Jul 15 11:05:15.620336 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.620319    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.623533 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.623507    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:05:15.623533 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.623530    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs\""
Jul 15 11:05:15.623705 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.623508    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:05:15.623705 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.623565    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:05:15.628707 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.628667    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"]
Jul 15 11:05:15.701053 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.701009    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkmrw\" (UniqueName: \"kubernetes.io/projected/bfe85d77-6bf2-4420-939c-21194d883c2c-kube-api-access-hkmrw\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.701053 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.701055    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.701271 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.701092    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfe85d77-6bf2-4420-939c-21194d883c2c-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.701271 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.701117    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.701271 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.701135    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.701271 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.701201    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.802165 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.802125    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.802165 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.802164    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.802442 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.802200    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.802442 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.802243    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkmrw\" (UniqueName: \"kubernetes.io/projected/bfe85d77-6bf2-4420-939c-21194d883c2c-kube-api-access-hkmrw\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.802442 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.802264    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.802593 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.802433    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfe85d77-6bf2-4420-939c-21194d883c2c-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.802645 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.802621    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.802700 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.802657    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.802700 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.802685    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.804789 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.804769    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.804953 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.804935    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfe85d77-6bf2-4420-939c-21194d883c2c-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.811497 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.811474    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkmrw\" (UniqueName: \"kubernetes.io/projected/bfe85d77-6bf2-4420-939c-21194d883c2c-kube-api-access-hkmrw\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:15.932536 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:15.932447    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:16.069406 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:16.069380    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"]
Jul 15 11:05:16.071951 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:05:16.071925    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe85d77_6bf2_4420_939c_21194d883c2c.slice/crio-8bab1ad1aeb08a6eec075fb866d08816b8d7a3d552c5713e6d56843b0e333369 WatchSource:0}: Error finding container 8bab1ad1aeb08a6eec075fb866d08816b8d7a3d552c5713e6d56843b0e333369: Status 404 returned error can't find the container with id 8bab1ad1aeb08a6eec075fb866d08816b8d7a3d552c5713e6d56843b0e333369
Jul 15 11:05:16.670112 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:16.670077    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt" event={"ID":"bfe85d77-6bf2-4420-939c-21194d883c2c","Type":"ContainerStarted","Data":"e51df44a0ea6561ed89ab368d3eca040cf88d8391dc8b47ff48f28c6ee34fd49"}
Jul 15 11:05:16.670526 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:16.670132    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt" event={"ID":"bfe85d77-6bf2-4420-939c-21194d883c2c","Type":"ContainerStarted","Data":"8bab1ad1aeb08a6eec075fb866d08816b8d7a3d552c5713e6d56843b0e333369"}
Jul 15 11:05:16.692626 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:16.692571    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt" podStartSLOduration=1.692555206 podStartE2EDuration="1.692555206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:05:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:05:16.690311615 +0000 UTC m=+1194.489785141" watchObservedRunningTime="2026-07-15 11:05:16.692555206 +0000 UTC m=+1194.492028787"
Jul 15 11:05:16.932930 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:16.932839    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:25.932968 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:25.932919    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:25.945586 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:25.945559    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:05:26.729248 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:05:26.729221    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:06:57.970437 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:06:57.970401    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"]
Jul 15 11:07:12.991710 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:12.991652    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt" podUID="bfe85d77-6bf2-4420-939c-21194d883c2c" containerName="main" containerID="cri-o://e51df44a0ea6561ed89ab368d3eca040cf88d8391dc8b47ff48f28c6ee34fd49" gracePeriod=45
Jul 15 11:07:13.205209 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.205180    2536 generic.go:356] "Generic (PLEG): container finished" podID="bfe85d77-6bf2-4420-939c-21194d883c2c" containerID="e51df44a0ea6561ed89ab368d3eca040cf88d8391dc8b47ff48f28c6ee34fd49" exitCode=0
Jul 15 11:07:13.205509 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.205217    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt" event={"ID":"bfe85d77-6bf2-4420-939c-21194d883c2c","Type":"ContainerDied","Data":"e51df44a0ea6561ed89ab368d3eca040cf88d8391dc8b47ff48f28c6ee34fd49"}
Jul 15 11:07:13.247116 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.247061    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:07:13.325548 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.325517    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-model-cache\") pod \"bfe85d77-6bf2-4420-939c-21194d883c2c\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") "
Jul 15 11:07:13.325721 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.325582    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkmrw\" (UniqueName: \"kubernetes.io/projected/bfe85d77-6bf2-4420-939c-21194d883c2c-kube-api-access-hkmrw\") pod \"bfe85d77-6bf2-4420-939c-21194d883c2c\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") "
Jul 15 11:07:13.325721 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.325630    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-dshm\") pod \"bfe85d77-6bf2-4420-939c-21194d883c2c\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") "
Jul 15 11:07:13.325721 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.325659    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-home\") pod \"bfe85d77-6bf2-4420-939c-21194d883c2c\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") "
Jul 15 11:07:13.325721 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.325686    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfe85d77-6bf2-4420-939c-21194d883c2c-tls-certs\") pod \"bfe85d77-6bf2-4420-939c-21194d883c2c\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") "
Jul 15 11:07:13.325721 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.325708    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-tmp-dir\") pod \"bfe85d77-6bf2-4420-939c-21194d883c2c\" (UID: \"bfe85d77-6bf2-4420-939c-21194d883c2c\") "
Jul 15 11:07:13.325938 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.325817    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-model-cache" (OuterVolumeSpecName: "model-cache") pod "bfe85d77-6bf2-4420-939c-21194d883c2c" (UID: "bfe85d77-6bf2-4420-939c-21194d883c2c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:07:13.325938 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.325890    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-home" (OuterVolumeSpecName: "home") pod "bfe85d77-6bf2-4420-939c-21194d883c2c" (UID: "bfe85d77-6bf2-4420-939c-21194d883c2c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:07:13.326052 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.325998    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:07:13.326106 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.326052    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:07:13.326106 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.326064    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bfe85d77-6bf2-4420-939c-21194d883c2c" (UID: "bfe85d77-6bf2-4420-939c-21194d883c2c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:07:13.327978 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.327952    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfe85d77-6bf2-4420-939c-21194d883c2c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bfe85d77-6bf2-4420-939c-21194d883c2c" (UID: "bfe85d77-6bf2-4420-939c-21194d883c2c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:07:13.327978 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.327960    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfe85d77-6bf2-4420-939c-21194d883c2c-kube-api-access-hkmrw" (OuterVolumeSpecName: "kube-api-access-hkmrw") pod "bfe85d77-6bf2-4420-939c-21194d883c2c" (UID: "bfe85d77-6bf2-4420-939c-21194d883c2c"). InnerVolumeSpecName "kube-api-access-hkmrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:07:13.328151 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.328005    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-dshm" (OuterVolumeSpecName: "dshm") pod "bfe85d77-6bf2-4420-939c-21194d883c2c" (UID: "bfe85d77-6bf2-4420-939c-21194d883c2c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:07:13.427495 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.427451    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkmrw\" (UniqueName: \"kubernetes.io/projected/bfe85d77-6bf2-4420-939c-21194d883c2c-kube-api-access-hkmrw\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:07:13.427495 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.427493    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:07:13.427495 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.427503    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfe85d77-6bf2-4420-939c-21194d883c2c-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:07:13.427495 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:13.427513    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfe85d77-6bf2-4420-939c-21194d883c2c-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:07:14.211159 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:14.211128    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"
Jul 15 11:07:14.211159 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:14.211153    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt" event={"ID":"bfe85d77-6bf2-4420-939c-21194d883c2c","Type":"ContainerDied","Data":"8bab1ad1aeb08a6eec075fb866d08816b8d7a3d552c5713e6d56843b0e333369"}
Jul 15 11:07:14.211671 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:14.211207    2536 scope.go:117] "RemoveContainer" containerID="e51df44a0ea6561ed89ab368d3eca040cf88d8391dc8b47ff48f28c6ee34fd49"
Jul 15 11:07:14.233612 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:14.233577    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"]
Jul 15 11:07:14.237187 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:14.237160    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-bxbpt"]
Jul 15 11:07:14.594943 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:07:14.594859    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfe85d77-6bf2-4420-939c-21194d883c2c" path="/var/lib/kubelet/pods/bfe85d77-6bf2-4420-939c-21194d883c2c/volumes"
Jul 15 11:09:07.830475 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.830433    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"]
Jul 15 11:09:07.831054 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.830794    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfe85d77-6bf2-4420-939c-21194d883c2c" containerName="main"
Jul 15 11:09:07.831054 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.830805    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfe85d77-6bf2-4420-939c-21194d883c2c" containerName="main"
Jul 15 11:09:07.831054 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.830871    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfe85d77-6bf2-4420-939c-21194d883c2c" containerName="main"
Jul 15 11:09:07.834100 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.834080    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:07.837485 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.837468    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:09:07.838011 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.837994    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs\""
Jul 15 11:09:07.838117 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.838097    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:09:07.838186 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.838106    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:09:07.844912 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.844891    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"]
Jul 15 11:09:07.912308 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.912258    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:07.912466 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.912368    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:07.912466 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.912450    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b1a52bd-5fff-47b6-b3bc-64318b992588-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:07.912603 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.912499    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrstv\" (UniqueName: \"kubernetes.io/projected/2b1a52bd-5fff-47b6-b3bc-64318b992588-kube-api-access-lrstv\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:07.912603 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.912529    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:07.912603 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:07.912570    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.013758 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.013734    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.013888 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.013779    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.013888 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.013802    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.013888 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.013834    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b1a52bd-5fff-47b6-b3bc-64318b992588-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.013888 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.013857    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrstv\" (UniqueName: \"kubernetes.io/projected/2b1a52bd-5fff-47b6-b3bc-64318b992588-kube-api-access-lrstv\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.013888 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.013874    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.014167 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.014132    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.014231 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.014196    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.014231 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.014215    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.016107 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.016088    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.016408 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.016390    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b1a52bd-5fff-47b6-b3bc-64318b992588-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.022553 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.022530    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrstv\" (UniqueName: \"kubernetes.io/projected/2b1a52bd-5fff-47b6-b3bc-64318b992588-kube-api-access-lrstv\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") " pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.145135 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.145069    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:08.285445 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.285413    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"]
Jul 15 11:09:08.287898 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:09:08.287866    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1a52bd_5fff_47b6_b3bc_64318b992588.slice/crio-4beb45c260aadcd05ccd4df2a9d303da8d344499e935478d2bf74ce0c77ff44a WatchSource:0}: Error finding container 4beb45c260aadcd05ccd4df2a9d303da8d344499e935478d2bf74ce0c77ff44a: Status 404 returned error can't find the container with id 4beb45c260aadcd05ccd4df2a9d303da8d344499e935478d2bf74ce0c77ff44a
Jul 15 11:09:08.728025 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.727995    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr" event={"ID":"2b1a52bd-5fff-47b6-b3bc-64318b992588","Type":"ContainerStarted","Data":"fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e"}
Jul 15 11:09:08.728187 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.728044    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr" event={"ID":"2b1a52bd-5fff-47b6-b3bc-64318b992588","Type":"ContainerStarted","Data":"4beb45c260aadcd05ccd4df2a9d303da8d344499e935478d2bf74ce0c77ff44a"}
Jul 15 11:09:08.747291 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:08.747238    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr" podStartSLOduration=1.747223753 podStartE2EDuration="1.747223753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:09:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:09:08.745078925 +0000 UTC m=+1426.544552452" watchObservedRunningTime="2026-07-15 11:09:08.747223753 +0000 UTC m=+1426.546697278"
Jul 15 11:09:09.146048 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:09.145981    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:18.146183 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:18.146151    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:18.158453 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:18.158427    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:18.785222 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:18.785148    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:30.191326 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:30.191293    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"]
Jul 15 11:09:45.211381 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.211332    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr" podUID="2b1a52bd-5fff-47b6-b3bc-64318b992588" containerName="main" containerID="cri-o://fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e" gracePeriod=45
Jul 15 11:09:45.460809 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.460782    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:45.508610 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.508548    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-dshm\") pod \"2b1a52bd-5fff-47b6-b3bc-64318b992588\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") "
Jul 15 11:09:45.508610 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.508590    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b1a52bd-5fff-47b6-b3bc-64318b992588-tls-certs\") pod \"2b1a52bd-5fff-47b6-b3bc-64318b992588\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") "
Jul 15 11:09:45.508786 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.508648    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-model-cache\") pod \"2b1a52bd-5fff-47b6-b3bc-64318b992588\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") "
Jul 15 11:09:45.508786 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.508688    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-tmp-dir\") pod \"2b1a52bd-5fff-47b6-b3bc-64318b992588\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") "
Jul 15 11:09:45.508896 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.508821    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrstv\" (UniqueName: \"kubernetes.io/projected/2b1a52bd-5fff-47b6-b3bc-64318b992588-kube-api-access-lrstv\") pod \"2b1a52bd-5fff-47b6-b3bc-64318b992588\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") "
Jul 15 11:09:45.508896 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.508870    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-home\") pod \"2b1a52bd-5fff-47b6-b3bc-64318b992588\" (UID: \"2b1a52bd-5fff-47b6-b3bc-64318b992588\") "
Jul 15 11:09:45.508989 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.508908    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b1a52bd-5fff-47b6-b3bc-64318b992588" (UID: "2b1a52bd-5fff-47b6-b3bc-64318b992588"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:09:45.508989 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.508920    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b1a52bd-5fff-47b6-b3bc-64318b992588" (UID: "2b1a52bd-5fff-47b6-b3bc-64318b992588"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:09:45.509142 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.509117    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-home" (OuterVolumeSpecName: "home") pod "2b1a52bd-5fff-47b6-b3bc-64318b992588" (UID: "2b1a52bd-5fff-47b6-b3bc-64318b992588"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:09:45.509337 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.509270    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:09:45.509589 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.509396    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:09:45.509589 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.509410    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:09:45.510769 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.510750    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-dshm" (OuterVolumeSpecName: "dshm") pod "2b1a52bd-5fff-47b6-b3bc-64318b992588" (UID: "2b1a52bd-5fff-47b6-b3bc-64318b992588"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:09:45.510903 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.510887    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b1a52bd-5fff-47b6-b3bc-64318b992588-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b1a52bd-5fff-47b6-b3bc-64318b992588" (UID: "2b1a52bd-5fff-47b6-b3bc-64318b992588"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:09:45.510957 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.510938    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b1a52bd-5fff-47b6-b3bc-64318b992588-kube-api-access-lrstv" (OuterVolumeSpecName: "kube-api-access-lrstv") pod "2b1a52bd-5fff-47b6-b3bc-64318b992588" (UID: "2b1a52bd-5fff-47b6-b3bc-64318b992588"). InnerVolumeSpecName "kube-api-access-lrstv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:09:45.610565 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.610533    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrstv\" (UniqueName: \"kubernetes.io/projected/2b1a52bd-5fff-47b6-b3bc-64318b992588-kube-api-access-lrstv\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:09:45.610565 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.610565    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b1a52bd-5fff-47b6-b3bc-64318b992588-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:09:45.610565 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.610574    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b1a52bd-5fff-47b6-b3bc-64318b992588-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:09:45.896068 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.896006    2536 generic.go:356] "Generic (PLEG): container finished" podID="2b1a52bd-5fff-47b6-b3bc-64318b992588" containerID="fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e" exitCode=0
Jul 15 11:09:45.896068 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.896057    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"
Jul 15 11:09:45.896218 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.896084    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr" event={"ID":"2b1a52bd-5fff-47b6-b3bc-64318b992588","Type":"ContainerDied","Data":"fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e"}
Jul 15 11:09:45.896218 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.896131    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr" event={"ID":"2b1a52bd-5fff-47b6-b3bc-64318b992588","Type":"ContainerDied","Data":"4beb45c260aadcd05ccd4df2a9d303da8d344499e935478d2bf74ce0c77ff44a"}
Jul 15 11:09:45.896218 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.896153    2536 scope.go:117] "RemoveContainer" containerID="fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e"
Jul 15 11:09:45.904790 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.904772    2536 scope.go:117] "RemoveContainer" containerID="fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e"
Jul 15 11:09:45.905048 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:09:45.905023    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e\": container with ID starting with fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e not found: ID does not exist" containerID="fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e"
Jul 15 11:09:45.905104 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.905056    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e"} err="failed to get container status \"fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e\": rpc error: code = NotFound desc = could not find container \"fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e\": container with ID starting with fb8e6d455ddc722f8f418d6d5052ecd3555ff74d6243fb49df49dedf3d70a71e not found: ID does not exist"
Jul 15 11:09:45.917101 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.917080    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"]
Jul 15 11:09:45.921765 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:45.921746    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4lvglr"]
Jul 15 11:09:46.595492 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:09:46.595463    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b1a52bd-5fff-47b6-b3bc-64318b992588" path="/var/lib/kubelet/pods/2b1a52bd-5fff-47b6-b3bc-64318b992588/volumes"
Jul 15 11:18:51.932067 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:51.932032    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"]
Jul 15 11:18:51.932555 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:51.932387    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b1a52bd-5fff-47b6-b3bc-64318b992588" containerName="main"
Jul 15 11:18:51.932555 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:51.932398    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b1a52bd-5fff-47b6-b3bc-64318b992588" containerName="main"
Jul 15 11:18:51.932555 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:51.932453    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b1a52bd-5fff-47b6-b3bc-64318b992588" containerName="main"
Jul 15 11:18:51.935330 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:51.935307    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:51.938566 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:51.938540    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs\""
Jul 15 11:18:51.938688 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:51.938544    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:18:51.938688 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:51.938606    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-pvc-router-manage-59b9d263-epp-sa-dockercfg-gckxg\""
Jul 15 11:18:51.938688 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:51.938658    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:18:51.938854 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:51.938823    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:18:51.944875 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:51.944851    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"]
Jul 15 11:18:52.036616 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.036589    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e008c0c-8773-40fb-a399-8de600deba58-tls-certs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:52.036738 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.036626    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngxjl\" (UniqueName: \"kubernetes.io/projected/0e008c0c-8773-40fb-a399-8de600deba58-kube-api-access-ngxjl\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:52.036738 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.036651    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0e008c0c-8773-40fb-a399-8de600deba58-tokenizer-uds\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:52.137807 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.137777    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0e008c0c-8773-40fb-a399-8de600deba58-tokenizer-uds\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:52.137946 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.137873    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e008c0c-8773-40fb-a399-8de600deba58-tls-certs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:52.137946 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.137919    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngxjl\" (UniqueName: \"kubernetes.io/projected/0e008c0c-8773-40fb-a399-8de600deba58-kube-api-access-ngxjl\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:52.138128 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.138109    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0e008c0c-8773-40fb-a399-8de600deba58-tokenizer-uds\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:52.140527 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.140494    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e008c0c-8773-40fb-a399-8de600deba58-tls-certs\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:52.146476 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.146455    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngxjl\" (UniqueName: \"kubernetes.io/projected/0e008c0c-8773-40fb-a399-8de600deba58-kube-api-access-ngxjl\") pod \"llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") " pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:52.245592 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.245559    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:52.377701 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.377677    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"]
Jul 15 11:18:52.380414 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:18:52.380389    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e008c0c_8773_40fb_a399_8de600deba58.slice/crio-c1bd533ab8239b75bfcc4cd403ddbc7253badef0283b1bcfd027e90f5efa18b5 WatchSource:0}: Error finding container c1bd533ab8239b75bfcc4cd403ddbc7253badef0283b1bcfd027e90f5efa18b5: Status 404 returned error can't find the container with id c1bd533ab8239b75bfcc4cd403ddbc7253badef0283b1bcfd027e90f5efa18b5
Jul 15 11:18:52.382353 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:52.382337    2536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 11:18:53.359464 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:53.359431    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw" event={"ID":"0e008c0c-8773-40fb-a399-8de600deba58","Type":"ContainerStarted","Data":"c1bd533ab8239b75bfcc4cd403ddbc7253badef0283b1bcfd027e90f5efa18b5"}
Jul 15 11:18:56.378685 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:56.378647    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw" event={"ID":"0e008c0c-8773-40fb-a399-8de600deba58","Type":"ContainerStarted","Data":"01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc"}
Jul 15 11:18:56.379135 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:56.378773    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:18:56.398263 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:18:56.398219    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw" podStartSLOduration=2.339702805 podStartE2EDuration="5.39820549s" totalImagesPullingTime="3.058502685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:18:56.39656079 +0000 UTC m=+2014.196034315" watchObservedRunningTime="2026-07-15 11:18:56.39820549 +0000 UTC m=+2014.197679014"
Jul 15 11:19:27.385521 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:19:27.385482    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:21:53.695741 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:21:53.695699    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"]
Jul 15 11:21:57.385631 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:21:57.385587    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw" podUID="0e008c0c-8773-40fb-a399-8de600deba58" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:22:07.385330 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:07.385292    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw" podUID="0e008c0c-8773-40fb-a399-8de600deba58" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:22:08.717661 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:08.717624    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw" podUID="0e008c0c-8773-40fb-a399-8de600deba58" containerName="main" containerID="cri-o://01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc" gracePeriod=45
Jul 15 11:22:08.969993 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:08.969936    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:22:09.109018 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.108969    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0e008c0c-8773-40fb-a399-8de600deba58-tokenizer-uds\") pod \"0e008c0c-8773-40fb-a399-8de600deba58\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") "
Jul 15 11:22:09.109191 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.109042    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e008c0c-8773-40fb-a399-8de600deba58-tls-certs\") pod \"0e008c0c-8773-40fb-a399-8de600deba58\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") "
Jul 15 11:22:09.109191 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.109082    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngxjl\" (UniqueName: \"kubernetes.io/projected/0e008c0c-8773-40fb-a399-8de600deba58-kube-api-access-ngxjl\") pod \"0e008c0c-8773-40fb-a399-8de600deba58\" (UID: \"0e008c0c-8773-40fb-a399-8de600deba58\") "
Jul 15 11:22:09.109326 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.109299    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e008c0c-8773-40fb-a399-8de600deba58-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "0e008c0c-8773-40fb-a399-8de600deba58" (UID: "0e008c0c-8773-40fb-a399-8de600deba58"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:22:09.111306 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.111262    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e008c0c-8773-40fb-a399-8de600deba58-kube-api-access-ngxjl" (OuterVolumeSpecName: "kube-api-access-ngxjl") pod "0e008c0c-8773-40fb-a399-8de600deba58" (UID: "0e008c0c-8773-40fb-a399-8de600deba58"). InnerVolumeSpecName "kube-api-access-ngxjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:22:09.111431 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.111353    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e008c0c-8773-40fb-a399-8de600deba58-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0e008c0c-8773-40fb-a399-8de600deba58" (UID: "0e008c0c-8773-40fb-a399-8de600deba58"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:22:09.209981 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.209955    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngxjl\" (UniqueName: \"kubernetes.io/projected/0e008c0c-8773-40fb-a399-8de600deba58-kube-api-access-ngxjl\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:22:09.209981 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.209977    2536 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/0e008c0c-8773-40fb-a399-8de600deba58-tokenizer-uds\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:22:09.210131 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.209987    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e008c0c-8773-40fb-a399-8de600deba58-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:22:09.237035 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.236968    2536 generic.go:356] "Generic (PLEG): container finished" podID="0e008c0c-8773-40fb-a399-8de600deba58" containerID="01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc" exitCode=0
Jul 15 11:22:09.237035 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.237010    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"
Jul 15 11:22:09.237035 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.237022    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw" event={"ID":"0e008c0c-8773-40fb-a399-8de600deba58","Type":"ContainerDied","Data":"01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc"}
Jul 15 11:22:09.237235 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.237060    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw" event={"ID":"0e008c0c-8773-40fb-a399-8de600deba58","Type":"ContainerDied","Data":"c1bd533ab8239b75bfcc4cd403ddbc7253badef0283b1bcfd027e90f5efa18b5"}
Jul 15 11:22:09.237235 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.237081    2536 scope.go:117] "RemoveContainer" containerID="01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc"
Jul 15 11:22:09.245838 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.245821    2536 scope.go:117] "RemoveContainer" containerID="01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc"
Jul 15 11:22:09.246099 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:22:09.246082    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc\": container with ID starting with 01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc not found: ID does not exist" containerID="01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc"
Jul 15 11:22:09.246144 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.246107    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc"} err="failed to get container status \"01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc\": rpc error: code = NotFound desc = could not find container \"01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc\": container with ID starting with 01e0b304055345f4b27520b664dd3af2f290f773f8eebf0b7a112d97489e70dc not found: ID does not exist"
Jul 15 11:22:09.258592 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.258570    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"]
Jul 15 11:22:09.262352 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:09.262333    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schemxlbw"]
Jul 15 11:22:10.595445 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:10.595411    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e008c0c-8773-40fb-a399-8de600deba58" path="/var/lib/kubelet/pods/0e008c0c-8773-40fb-a399-8de600deba58/volumes"
Jul 15 11:22:24.637899 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.637865    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"]
Jul 15 11:22:24.638509 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.638242    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e008c0c-8773-40fb-a399-8de600deba58" containerName="main"
Jul 15 11:22:24.638509 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.638259    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e008c0c-8773-40fb-a399-8de600deba58" containerName="main"
Jul 15 11:22:24.638509 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.638342    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e008c0c-8773-40fb-a399-8de600deba58" containerName="main"
Jul 15 11:22:24.642495 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.642479    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:24.645634 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.645610    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:22:24.645634 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.645627    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs\""
Jul 15 11:22:24.645926 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.645620    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:22:24.645926 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.645895    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:22:24.646300 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.646256    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-pvc-router-manage-e8706282-epp-sa-dockercfg-w7456\""
Jul 15 11:22:24.650414 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.650397    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"]
Jul 15 11:22:24.745783 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.745750    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dqcq\" (UniqueName: \"kubernetes.io/projected/e7952ab8-cf3b-4074-8177-d03bbc7593d0-kube-api-access-7dqcq\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:24.745978 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.745922    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:24.745978 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.745951    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tokenizer-uds\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:24.846783 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.846749    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:24.846783 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.846783    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tokenizer-uds\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:24.847000 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.846809    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dqcq\" (UniqueName: \"kubernetes.io/projected/e7952ab8-cf3b-4074-8177-d03bbc7593d0-kube-api-access-7dqcq\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:24.847143 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.847123    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tokenizer-uds\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:24.849500 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.849477    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tls-certs\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:24.855359 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.855339    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dqcq\" (UniqueName: \"kubernetes.io/projected/e7952ab8-cf3b-4074-8177-d03bbc7593d0-kube-api-access-7dqcq\") pod \"llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") " pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:24.952169 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:24.952083    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:25.090636 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:25.090610    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"]
Jul 15 11:22:25.092708 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:22:25.092678    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7952ab8_cf3b_4074_8177_d03bbc7593d0.slice/crio-95a9cb5d0dec5f5d4ce7308e1cf4fc62098dae9b6de42c09a4ae9cf593007ff5 WatchSource:0}: Error finding container 95a9cb5d0dec5f5d4ce7308e1cf4fc62098dae9b6de42c09a4ae9cf593007ff5: Status 404 returned error can't find the container with id 95a9cb5d0dec5f5d4ce7308e1cf4fc62098dae9b6de42c09a4ae9cf593007ff5
Jul 15 11:22:25.312323 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:25.312263    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd" event={"ID":"e7952ab8-cf3b-4074-8177-d03bbc7593d0","Type":"ContainerStarted","Data":"e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73"}
Jul 15 11:22:25.312517 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:25.312340    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd" event={"ID":"e7952ab8-cf3b-4074-8177-d03bbc7593d0","Type":"ContainerStarted","Data":"95a9cb5d0dec5f5d4ce7308e1cf4fc62098dae9b6de42c09a4ae9cf593007ff5"}
Jul 15 11:22:25.312517 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:25.312448    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:22:25.330493 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:25.330441    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd" podStartSLOduration=1.330424491 podStartE2EDuration="1.330424491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:22:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:22:25.328647612 +0000 UTC m=+2223.128121139" watchObservedRunningTime="2026-07-15 11:22:25.330424491 +0000 UTC m=+2223.129898015"
Jul 15 11:22:56.318360 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:22:56.318332    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:25:44.797495 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:25:44.797405    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"]
Jul 15 11:25:46.318453 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:25:46.318422    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd" podUID="e7952ab8-cf3b-4074-8177-d03bbc7593d0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:25:56.318170 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:25:56.318136    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd" podUID="e7952ab8-cf3b-4074-8177-d03bbc7593d0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:25:59.819235 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:25:59.819172    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd" podUID="e7952ab8-cf3b-4074-8177-d03bbc7593d0" containerName="main" containerID="cri-o://e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73" gracePeriod=45
Jul 15 11:26:00.070245 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.070189    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:26:00.166419 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.166389    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tokenizer-uds\") pod \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") "
Jul 15 11:26:00.166590 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.166461    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dqcq\" (UniqueName: \"kubernetes.io/projected/e7952ab8-cf3b-4074-8177-d03bbc7593d0-kube-api-access-7dqcq\") pod \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") "
Jul 15 11:26:00.166590 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.166511    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tls-certs\") pod \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\" (UID: \"e7952ab8-cf3b-4074-8177-d03bbc7593d0\") "
Jul 15 11:26:00.166721 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.166694    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "e7952ab8-cf3b-4074-8177-d03bbc7593d0" (UID: "e7952ab8-cf3b-4074-8177-d03bbc7593d0"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:26:00.168675 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.168646    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e7952ab8-cf3b-4074-8177-d03bbc7593d0" (UID: "e7952ab8-cf3b-4074-8177-d03bbc7593d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:26:00.168774 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.168684    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7952ab8-cf3b-4074-8177-d03bbc7593d0-kube-api-access-7dqcq" (OuterVolumeSpecName: "kube-api-access-7dqcq") pod "e7952ab8-cf3b-4074-8177-d03bbc7593d0" (UID: "e7952ab8-cf3b-4074-8177-d03bbc7593d0"). InnerVolumeSpecName "kube-api-access-7dqcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:26:00.260943 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.260910    2536 generic.go:356] "Generic (PLEG): container finished" podID="e7952ab8-cf3b-4074-8177-d03bbc7593d0" containerID="e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73" exitCode=0
Jul 15 11:26:00.261062 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.260959    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"
Jul 15 11:26:00.261062 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.261004    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd" event={"ID":"e7952ab8-cf3b-4074-8177-d03bbc7593d0","Type":"ContainerDied","Data":"e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73"}
Jul 15 11:26:00.261175 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.261062    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd" event={"ID":"e7952ab8-cf3b-4074-8177-d03bbc7593d0","Type":"ContainerDied","Data":"95a9cb5d0dec5f5d4ce7308e1cf4fc62098dae9b6de42c09a4ae9cf593007ff5"}
Jul 15 11:26:00.261175 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.261092    2536 scope.go:117] "RemoveContainer" containerID="e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73"
Jul 15 11:26:00.267308 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.267263    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dqcq\" (UniqueName: \"kubernetes.io/projected/e7952ab8-cf3b-4074-8177-d03bbc7593d0-kube-api-access-7dqcq\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:26:00.267308 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.267305    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:26:00.267461 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.267315    2536 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/e7952ab8-cf3b-4074-8177-d03bbc7593d0-tokenizer-uds\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:26:00.270329 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.270313    2536 scope.go:117] "RemoveContainer" containerID="e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73"
Jul 15 11:26:00.270588 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:26:00.270570    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73\": container with ID starting with e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73 not found: ID does not exist" containerID="e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73"
Jul 15 11:26:00.270653 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.270601    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73"} err="failed to get container status \"e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73\": rpc error: code = NotFound desc = could not find container \"e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73\": container with ID starting with e99ade86103098bec11d53933ff335d03783f0cf0cc8cd645b374247f04f0d73 not found: ID does not exist"
Jul 15 11:26:00.281884 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.281859    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"]
Jul 15 11:26:00.286662 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.286639    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheg6hpd"]
Jul 15 11:26:00.595502 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:00.595471    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7952ab8-cf3b-4074-8177-d03bbc7593d0" path="/var/lib/kubelet/pods/e7952ab8-cf3b-4074-8177-d03bbc7593d0/volumes"
Jul 15 11:26:18.213969 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.213914    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"]
Jul 15 11:26:18.214449 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.214311    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7952ab8-cf3b-4074-8177-d03bbc7593d0" containerName="main"
Jul 15 11:26:18.214449 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.214323    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7952ab8-cf3b-4074-8177-d03bbc7593d0" containerName="main"
Jul 15 11:26:18.214449 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.214396    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7952ab8-cf3b-4074-8177-d03bbc7593d0" containerName="main"
Jul 15 11:26:18.218687 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.218664    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:18.221639 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.221609    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:26:18.221760 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.221615    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:26:18.221760 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.221706    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:26:18.221760 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.221728    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs\""
Jul 15 11:26:18.221917 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.221685    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-pvc-router-manage-2577e794-epp-sa-dockercfg-psthn\""
Jul 15 11:26:18.228453 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.228434    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"]
Jul 15 11:26:18.312457 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.312428    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2wt6\" (UniqueName: \"kubernetes.io/projected/7fea5cee-08ae-4c3d-b903-d235c518f922-kube-api-access-t2wt6\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:18.312597 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.312467    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fea5cee-08ae-4c3d-b903-d235c518f922-tls-certs\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:18.312597 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.312534    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/7fea5cee-08ae-4c3d-b903-d235c518f922-tokenizer-uds\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:18.413266 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.413231    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/7fea5cee-08ae-4c3d-b903-d235c518f922-tokenizer-uds\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:18.413450 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.413359    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2wt6\" (UniqueName: \"kubernetes.io/projected/7fea5cee-08ae-4c3d-b903-d235c518f922-kube-api-access-t2wt6\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:18.413450 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.413399    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fea5cee-08ae-4c3d-b903-d235c518f922-tls-certs\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:18.413609 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.413587    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/7fea5cee-08ae-4c3d-b903-d235c518f922-tokenizer-uds\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:18.415936 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.415911    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fea5cee-08ae-4c3d-b903-d235c518f922-tls-certs\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:18.422138 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.422115    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2wt6\" (UniqueName: \"kubernetes.io/projected/7fea5cee-08ae-4c3d-b903-d235c518f922-kube-api-access-t2wt6\") pod \"llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") " pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:18.528770 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.528696    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:18.668061 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:18.668025    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"]
Jul 15 11:26:18.671147 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:26:18.671115    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fea5cee_08ae_4c3d_b903_d235c518f922.slice/crio-06065c6b0d080c94d41c0a3ffa79ca19b3c40f88a9c1cc62dceaea8809c8cffc WatchSource:0}: Error finding container 06065c6b0d080c94d41c0a3ffa79ca19b3c40f88a9c1cc62dceaea8809c8cffc: Status 404 returned error can't find the container with id 06065c6b0d080c94d41c0a3ffa79ca19b3c40f88a9c1cc62dceaea8809c8cffc
Jul 15 11:26:19.348014 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:19.347980    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq" event={"ID":"7fea5cee-08ae-4c3d-b903-d235c518f922","Type":"ContainerStarted","Data":"170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0"}
Jul 15 11:26:19.348408 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:19.348027    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq" event={"ID":"7fea5cee-08ae-4c3d-b903-d235c518f922","Type":"ContainerStarted","Data":"06065c6b0d080c94d41c0a3ffa79ca19b3c40f88a9c1cc62dceaea8809c8cffc"}
Jul 15 11:26:19.348408 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:19.348105    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:26:19.367760 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:19.367703    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq" podStartSLOduration=1.367682918 podStartE2EDuration="1.367682918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 11:26:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:26:19.366557004 +0000 UTC m=+2457.166030532" watchObservedRunningTime="2026-07-15 11:26:19.367682918 +0000 UTC m=+2457.167156444"
Jul 15 11:26:50.355434 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:26:50.355407    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:33:59.500849 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:33:59.500813    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"]
Jul 15 11:34:00.354749 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:00.354710    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq" podUID="7fea5cee-08ae-4c3d-b903-d235c518f922" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:34:10.354580 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:10.354529    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq" podUID="7fea5cee-08ae-4c3d-b903-d235c518f922" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:34:14.522135 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:14.522078    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq" podUID="7fea5cee-08ae-4c3d-b903-d235c518f922" containerName="main" containerID="cri-o://170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0" gracePeriod=45
Jul 15 11:34:14.769968 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:14.769939    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:34:14.846052 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:14.845976    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/7fea5cee-08ae-4c3d-b903-d235c518f922-tokenizer-uds\") pod \"7fea5cee-08ae-4c3d-b903-d235c518f922\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") "
Jul 15 11:34:14.846170 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:14.846076    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2wt6\" (UniqueName: \"kubernetes.io/projected/7fea5cee-08ae-4c3d-b903-d235c518f922-kube-api-access-t2wt6\") pod \"7fea5cee-08ae-4c3d-b903-d235c518f922\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") "
Jul 15 11:34:14.846170 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:14.846100    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fea5cee-08ae-4c3d-b903-d235c518f922-tls-certs\") pod \"7fea5cee-08ae-4c3d-b903-d235c518f922\" (UID: \"7fea5cee-08ae-4c3d-b903-d235c518f922\") "
Jul 15 11:34:14.846251 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:14.846226    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fea5cee-08ae-4c3d-b903-d235c518f922-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "7fea5cee-08ae-4c3d-b903-d235c518f922" (UID: "7fea5cee-08ae-4c3d-b903-d235c518f922"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:34:14.846369 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:14.846357    2536 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/7fea5cee-08ae-4c3d-b903-d235c518f922-tokenizer-uds\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:34:14.848294 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:14.848246    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fea5cee-08ae-4c3d-b903-d235c518f922-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7fea5cee-08ae-4c3d-b903-d235c518f922" (UID: "7fea5cee-08ae-4c3d-b903-d235c518f922"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:34:14.848391 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:14.848360    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fea5cee-08ae-4c3d-b903-d235c518f922-kube-api-access-t2wt6" (OuterVolumeSpecName: "kube-api-access-t2wt6") pod "7fea5cee-08ae-4c3d-b903-d235c518f922" (UID: "7fea5cee-08ae-4c3d-b903-d235c518f922"). InnerVolumeSpecName "kube-api-access-t2wt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:34:14.946900 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:14.946870    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2wt6\" (UniqueName: \"kubernetes.io/projected/7fea5cee-08ae-4c3d-b903-d235c518f922-kube-api-access-t2wt6\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:34:14.946900 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:14.946894    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fea5cee-08ae-4c3d-b903-d235c518f922-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:34:15.486736 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:15.486699    2536 generic.go:356] "Generic (PLEG): container finished" podID="7fea5cee-08ae-4c3d-b903-d235c518f922" containerID="170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0" exitCode=0
Jul 15 11:34:15.486912 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:15.486749    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"
Jul 15 11:34:15.486912 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:15.486774    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq" event={"ID":"7fea5cee-08ae-4c3d-b903-d235c518f922","Type":"ContainerDied","Data":"170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0"}
Jul 15 11:34:15.486912 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:15.486822    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq" event={"ID":"7fea5cee-08ae-4c3d-b903-d235c518f922","Type":"ContainerDied","Data":"06065c6b0d080c94d41c0a3ffa79ca19b3c40f88a9c1cc62dceaea8809c8cffc"}
Jul 15 11:34:15.486912 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:15.486849    2536 scope.go:117] "RemoveContainer" containerID="170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0"
Jul 15 11:34:15.496643 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:15.496572    2536 scope.go:117] "RemoveContainer" containerID="170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0"
Jul 15 11:34:15.497879 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:34:15.497845    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0\": container with ID starting with 170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0 not found: ID does not exist" containerID="170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0"
Jul 15 11:34:15.497879 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:15.497873    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0"} err="failed to get container status \"170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0\": rpc error: code = NotFound desc = could not find container \"170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0\": container with ID starting with 170f8da193e115c1f66a66110f52fd3bda987e61b21fd8efcd5130a6e3f4ade0 not found: ID does not exist"
Jul 15 11:34:15.508266 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:15.508234    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"]
Jul 15 11:34:15.512527 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:15.512502    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-sche8n4bq"]
Jul 15 11:34:16.594625 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:34:16.594591    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fea5cee-08ae-4c3d-b903-d235c518f922" path="/var/lib/kubelet/pods/7fea5cee-08ae-4c3d-b903-d235c518f922/volumes"
Jul 15 11:37:36.536113 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.536082    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"]
Jul 15 11:37:36.536578 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.536425    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fea5cee-08ae-4c3d-b903-d235c518f922" containerName="main"
Jul 15 11:37:36.536578 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.536437    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fea5cee-08ae-4c3d-b903-d235c518f922" containerName="main"
Jul 15 11:37:36.536578 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.536498    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fea5cee-08ae-4c3d-b903-d235c518f922" containerName="main"
Jul 15 11:37:36.539512 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.539496    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.543962 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.543942    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Jul 15 11:37:36.544245 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.544231    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:37:36.544491 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.544476    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:37:36.544713 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.544698    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:37:36.563089 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.563064    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"]
Jul 15 11:37:36.704504 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.704473    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-tmp-dir\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.704504 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.704504    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbc67\" (UniqueName: \"kubernetes.io/projected/fd675f38-e486-4585-9bc2-3b86a954b249-kube-api-access-gbc67\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.704695 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.704531    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd675f38-e486-4585-9bc2-3b86a954b249-tls-certs\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.704695 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.704555    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-model-cache\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.704695 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.704599    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-dshm\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.704795 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.704694    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-home\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.805495 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.805411    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-home\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.805495 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.805482    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-tmp-dir\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.805712 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.805506    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbc67\" (UniqueName: \"kubernetes.io/projected/fd675f38-e486-4585-9bc2-3b86a954b249-kube-api-access-gbc67\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.805712 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.805533    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd675f38-e486-4585-9bc2-3b86a954b249-tls-certs\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.805712 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.805552    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-model-cache\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.805712 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.805586    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-dshm\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.805923 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.805895    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-home\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.805979 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.805934    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-model-cache\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.805979 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.805968    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-tmp-dir\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.807938 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.807918    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-dshm\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.808162 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.808143    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd675f38-e486-4585-9bc2-3b86a954b249-tls-certs\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.814450 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.814433    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbc67\" (UniqueName: \"kubernetes.io/projected/fd675f38-e486-4585-9bc2-3b86a954b249-kube-api-access-gbc67\") pod \"scheduler-inline-config-test-kserve-7c6c58cc96-flhdx\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.849408 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.849387    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:36.922313 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.921679    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"]
Jul 15 11:37:36.926048 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.926029    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:36.929048 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.929029    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-epp-sa-dockercfg-hth82\""
Jul 15 11:37:36.936003 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.935980    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"]
Jul 15 11:37:36.993574 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:36.993551    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"]
Jul 15 11:37:36.995792 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:37:36.995761    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd675f38_e486_4585_9bc2_3b86a954b249.slice/crio-05595b22aa6f22c86be4640ab299df24cbcc7b2080e642ef02d157206ccad0be WatchSource:0}: Error finding container 05595b22aa6f22c86be4640ab299df24cbcc7b2080e642ef02d157206ccad0be: Status 404 returned error can't find the container with id 05595b22aa6f22c86be4640ab299df24cbcc7b2080e642ef02d157206ccad0be
Jul 15 11:37:37.108443 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.108351    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:37.108604 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.108527    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:37.108604 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.108563    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-799h8\" (UniqueName: \"kubernetes.io/projected/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-kube-api-access-799h8\") pod \"scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:37.209026 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.208996    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:37.209185 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.209031    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-799h8\" (UniqueName: \"kubernetes.io/projected/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-kube-api-access-799h8\") pod \"scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:37.209185 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.209166    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:37.209553 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.209536    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tokenizer-uds\") pod \"scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:37.211625 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.211609    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:37.219299 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.219252    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-799h8\" (UniqueName: \"kubernetes.io/projected/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-kube-api-access-799h8\") pod \"scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:37.237218 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.237200    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:37.367925 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.367848    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"]
Jul 15 11:37:37.371010 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:37:37.370984    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6ad054a_e51e_47a5_ac1d_ac2fd9d60072.slice/crio-e2536eeee0ebef971527bd404c65e05fe27fce5510637b7292389c4e906f76e2 WatchSource:0}: Error finding container e2536eeee0ebef971527bd404c65e05fe27fce5510637b7292389c4e906f76e2: Status 404 returned error can't find the container with id e2536eeee0ebef971527bd404c65e05fe27fce5510637b7292389c4e906f76e2
Jul 15 11:37:37.389705 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.389677    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4" event={"ID":"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072","Type":"ContainerStarted","Data":"e2536eeee0ebef971527bd404c65e05fe27fce5510637b7292389c4e906f76e2"}
Jul 15 11:37:37.391388 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.391364    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx" event={"ID":"fd675f38-e486-4585-9bc2-3b86a954b249","Type":"ContainerStarted","Data":"91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde"}
Jul 15 11:37:37.391502 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.391411    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx" event={"ID":"fd675f38-e486-4585-9bc2-3b86a954b249","Type":"ContainerStarted","Data":"05595b22aa6f22c86be4640ab299df24cbcc7b2080e642ef02d157206ccad0be"}
Jul 15 11:37:37.411881 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.411839    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx" podStartSLOduration=1.411823817 podStartE2EDuration="1.411823817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:37:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:37:37.409604906 +0000 UTC m=+3135.209078427" watchObservedRunningTime="2026-07-15 11:37:37.411823817 +0000 UTC m=+3135.211297341"
Jul 15 11:37:37.850162 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:37.850129    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:38.397688 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:38.397649    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4" event={"ID":"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072","Type":"ContainerStarted","Data":"3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4"}
Jul 15 11:37:38.416476 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:38.416425    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4" podStartSLOduration=2.416408969 podStartE2EDuration="2.416408969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:37:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:37:38.415759226 +0000 UTC m=+3136.215232753" watchObservedRunningTime="2026-07-15 11:37:38.416408969 +0000 UTC m=+3136.215882493"
Jul 15 11:37:39.403438 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:39.403409    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:37:46.849936 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:46.849846    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:46.862240 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:46.862215    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:37:47.450473 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:37:47.450438    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:38:10.408788 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:10.408758    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:38:11.586651 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:11.586621    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"]
Jul 15 11:38:11.588551 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:11.588521    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"]
Jul 15 11:38:20.409574 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:20.409536    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4" podUID="f6ad054a-e51e-47a5-ac1d-ac2fd9d60072" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:38:26.609505 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:26.609466    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx" podUID="fd675f38-e486-4585-9bc2-3b86a954b249" containerName="main" containerID="cri-o://91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde" gracePeriod=45
Jul 15 11:38:26.612666 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:26.612632    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4" podUID="f6ad054a-e51e-47a5-ac1d-ac2fd9d60072" containerName="main" containerID="cri-o://3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4" gracePeriod=45
Jul 15 11:38:26.876672 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:26.876641    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:38:26.881559 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:26.881539    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:38:27.017725 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.017700    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd675f38-e486-4585-9bc2-3b86a954b249-tls-certs\") pod \"fd675f38-e486-4585-9bc2-3b86a954b249\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") "
Jul 15 11:38:27.017871 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.017743    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-dshm\") pod \"fd675f38-e486-4585-9bc2-3b86a954b249\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") "
Jul 15 11:38:27.017871 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.017765    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbc67\" (UniqueName: \"kubernetes.io/projected/fd675f38-e486-4585-9bc2-3b86a954b249-kube-api-access-gbc67\") pod \"fd675f38-e486-4585-9bc2-3b86a954b249\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") "
Jul 15 11:38:27.017871 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.017798    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tls-certs\") pod \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") "
Jul 15 11:38:27.017871 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.017814    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-799h8\" (UniqueName: \"kubernetes.io/projected/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-kube-api-access-799h8\") pod \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") "
Jul 15 11:38:27.017871 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.017830    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-home\") pod \"fd675f38-e486-4585-9bc2-3b86a954b249\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") "
Jul 15 11:38:27.017871 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.017851    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tokenizer-uds\") pod \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\" (UID: \"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072\") "
Jul 15 11:38:27.018183 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.017880    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-tmp-dir\") pod \"fd675f38-e486-4585-9bc2-3b86a954b249\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") "
Jul 15 11:38:27.018183 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.017914    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-model-cache\") pod \"fd675f38-e486-4585-9bc2-3b86a954b249\" (UID: \"fd675f38-e486-4585-9bc2-3b86a954b249\") "
Jul 15 11:38:27.018183 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.018143    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-home" (OuterVolumeSpecName: "home") pod "fd675f38-e486-4585-9bc2-3b86a954b249" (UID: "fd675f38-e486-4585-9bc2-3b86a954b249"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:38:27.018398 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.018315    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "f6ad054a-e51e-47a5-ac1d-ac2fd9d60072" (UID: "f6ad054a-e51e-47a5-ac1d-ac2fd9d60072"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:38:27.018398 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.018332    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-model-cache" (OuterVolumeSpecName: "model-cache") pod "fd675f38-e486-4585-9bc2-3b86a954b249" (UID: "fd675f38-e486-4585-9bc2-3b86a954b249"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:38:27.018398 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.018375    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fd675f38-e486-4585-9bc2-3b86a954b249" (UID: "fd675f38-e486-4585-9bc2-3b86a954b249"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:38:27.020208 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.020184    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-kube-api-access-799h8" (OuterVolumeSpecName: "kube-api-access-799h8") pod "f6ad054a-e51e-47a5-ac1d-ac2fd9d60072" (UID: "f6ad054a-e51e-47a5-ac1d-ac2fd9d60072"). InnerVolumeSpecName "kube-api-access-799h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:38:27.020530 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.020502    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f6ad054a-e51e-47a5-ac1d-ac2fd9d60072" (UID: "f6ad054a-e51e-47a5-ac1d-ac2fd9d60072"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:38:27.020616 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.020521    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd675f38-e486-4585-9bc2-3b86a954b249-kube-api-access-gbc67" (OuterVolumeSpecName: "kube-api-access-gbc67") pod "fd675f38-e486-4585-9bc2-3b86a954b249" (UID: "fd675f38-e486-4585-9bc2-3b86a954b249"). InnerVolumeSpecName "kube-api-access-gbc67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:38:27.020616 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.020567    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd675f38-e486-4585-9bc2-3b86a954b249-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fd675f38-e486-4585-9bc2-3b86a954b249" (UID: "fd675f38-e486-4585-9bc2-3b86a954b249"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:38:27.020616 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.020574    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-dshm" (OuterVolumeSpecName: "dshm") pod "fd675f38-e486-4585-9bc2-3b86a954b249" (UID: "fd675f38-e486-4585-9bc2-3b86a954b249"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:38:27.118784 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.118732    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd675f38-e486-4585-9bc2-3b86a954b249-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:38:27.118784 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.118752    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:38:27.118784 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.118760    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbc67\" (UniqueName: \"kubernetes.io/projected/fd675f38-e486-4585-9bc2-3b86a954b249-kube-api-access-gbc67\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:38:27.118784 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.118769    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:38:27.118784 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.118778    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-799h8\" (UniqueName: \"kubernetes.io/projected/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-kube-api-access-799h8\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:38:27.118991 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.118788    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:38:27.118991 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.118796    2536 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072-tokenizer-uds\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:38:27.118991 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.118804    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:38:27.118991 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.118811    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd675f38-e486-4585-9bc2-3b86a954b249-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:38:27.623091 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.623053    2536 generic.go:356] "Generic (PLEG): container finished" podID="fd675f38-e486-4585-9bc2-3b86a954b249" containerID="91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde" exitCode=0
Jul 15 11:38:27.623560 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.623095    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"
Jul 15 11:38:27.623560 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.623138    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx" event={"ID":"fd675f38-e486-4585-9bc2-3b86a954b249","Type":"ContainerDied","Data":"91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde"}
Jul 15 11:38:27.623560 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.623191    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx" event={"ID":"fd675f38-e486-4585-9bc2-3b86a954b249","Type":"ContainerDied","Data":"05595b22aa6f22c86be4640ab299df24cbcc7b2080e642ef02d157206ccad0be"}
Jul 15 11:38:27.623560 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.623222    2536 scope.go:117] "RemoveContainer" containerID="91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde"
Jul 15 11:38:27.624788 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.624677    2536 generic.go:356] "Generic (PLEG): container finished" podID="f6ad054a-e51e-47a5-ac1d-ac2fd9d60072" containerID="3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4" exitCode=0
Jul 15 11:38:27.624788 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.624721    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"
Jul 15 11:38:27.624788 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.624727    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4" event={"ID":"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072","Type":"ContainerDied","Data":"3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4"}
Jul 15 11:38:27.624788 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.624761    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4" event={"ID":"f6ad054a-e51e-47a5-ac1d-ac2fd9d60072","Type":"ContainerDied","Data":"e2536eeee0ebef971527bd404c65e05fe27fce5510637b7292389c4e906f76e2"}
Jul 15 11:38:27.632472 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.632453    2536 scope.go:117] "RemoveContainer" containerID="91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde"
Jul 15 11:38:27.632752 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:38:27.632709    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde\": container with ID starting with 91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde not found: ID does not exist" containerID="91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde"
Jul 15 11:38:27.632752 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.632734    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde"} err="failed to get container status \"91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde\": rpc error: code = NotFound desc = could not find container \"91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde\": container with ID starting with 91c411f34e5d4567ade3b4f789dcbf29593590c1465fc1cf7277c1abfe621cde not found: ID does not exist"
Jul 15 11:38:27.632914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.632757    2536 scope.go:117] "RemoveContainer" containerID="3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4"
Jul 15 11:38:27.640414 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.640397    2536 scope.go:117] "RemoveContainer" containerID="3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4"
Jul 15 11:38:27.640655 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:38:27.640636    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4\": container with ID starting with 3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4 not found: ID does not exist" containerID="3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4"
Jul 15 11:38:27.640718 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.640665    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4"} err="failed to get container status \"3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4\": rpc error: code = NotFound desc = could not find container \"3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4\": container with ID starting with 3e0508676947979c1e662e255dd6b1784c881085b0c6356d329d356f0e22f2d4 not found: ID does not exist"
Jul 15 11:38:27.674178 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.674150    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"]
Jul 15 11:38:27.688831 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.688806    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-flhdx"]
Jul 15 11:38:27.708649 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.708625    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"]
Jul 15 11:38:27.716364 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:27.716345    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-568fcsmrs4"]
Jul 15 11:38:28.595129 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:28.595101    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6ad054a-e51e-47a5-ac1d-ac2fd9d60072" path="/var/lib/kubelet/pods/f6ad054a-e51e-47a5-ac1d-ac2fd9d60072/volumes"
Jul 15 11:38:28.595478 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:28.595465    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd675f38-e486-4585-9bc2-3b86a954b249" path="/var/lib/kubelet/pods/fd675f38-e486-4585-9bc2-3b86a954b249/volumes"
Jul 15 11:38:38.123904 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.123875    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"]
Jul 15 11:38:38.124256 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.124181    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd675f38-e486-4585-9bc2-3b86a954b249" containerName="main"
Jul 15 11:38:38.124256 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.124192    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd675f38-e486-4585-9bc2-3b86a954b249" containerName="main"
Jul 15 11:38:38.124256 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.124201    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6ad054a-e51e-47a5-ac1d-ac2fd9d60072" containerName="main"
Jul 15 11:38:38.124256 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.124206    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6ad054a-e51e-47a5-ac1d-ac2fd9d60072" containerName="main"
Jul 15 11:38:38.124409 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.124268    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6ad054a-e51e-47a5-ac1d-ac2fd9d60072" containerName="main"
Jul 15 11:38:38.124409 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.124291    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd675f38-e486-4585-9bc2-3b86a954b249" containerName="main"
Jul 15 11:38:38.127260 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.127244    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.130675 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.130656    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:38:38.130761 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.130690    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:38:38.130809 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.130692    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\""
Jul 15 11:38:38.130908 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.130895    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:38:38.136645 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.136622    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"]
Jul 15 11:38:38.303821 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.303789    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.303821 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.303825    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.304028 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.303850    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.304028 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.303950    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36a4a645-3220-4c38-9259-02a9eff0c6d5-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.304028 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.304012    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.304139 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.304065    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqzxh\" (UniqueName: \"kubernetes.io/projected/36a4a645-3220-4c38-9259-02a9eff0c6d5-kube-api-access-qqzxh\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.405141 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.405074    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.405141 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.405113    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.405332 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.405146    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.405332 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.405195    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36a4a645-3220-4c38-9259-02a9eff0c6d5-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.405449 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.405346    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.405449 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.405408    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqzxh\" (UniqueName: \"kubernetes.io/projected/36a4a645-3220-4c38-9259-02a9eff0c6d5-kube-api-access-qqzxh\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.405552 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.405475    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.405552 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.405531    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.405657 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.405586    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.407537 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.407521    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.407834 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.407813    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36a4a645-3220-4c38-9259-02a9eff0c6d5-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.415067 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.415042    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqzxh\" (UniqueName: \"kubernetes.io/projected/36a4a645-3220-4c38-9259-02a9eff0c6d5-kube-api-access-qqzxh\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") " pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.438073 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.438054    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:38.533295 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.533243    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"]
Jul 15 11:38:38.536923 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.536901    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:38.539779 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.539758    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa-dockercfg-94kc6\""
Jul 15 11:38:38.549130 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.549101    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"]
Jul 15 11:38:38.592540 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:38:38.592512    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a4a645_3220_4c38_9259_02a9eff0c6d5.slice/crio-b6a11c139c712a175109624dcad54dd60ee506cc2e5526da540e53ab5f945bf2 WatchSource:0}: Error finding container b6a11c139c712a175109624dcad54dd60ee506cc2e5526da540e53ab5f945bf2: Status 404 returned error can't find the container with id b6a11c139c712a175109624dcad54dd60ee506cc2e5526da540e53ab5f945bf2
Jul 15 11:38:38.595570 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.595549    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"]
Jul 15 11:38:38.673385 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.673321    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4" event={"ID":"36a4a645-3220-4c38-9259-02a9eff0c6d5","Type":"ContainerStarted","Data":"49fec28b5de106e809aeb291bdef7d6e5a51de5d609b04d7b3429701d74f5104"}
Jul 15 11:38:38.673385 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.673377    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4" event={"ID":"36a4a645-3220-4c38-9259-02a9eff0c6d5","Type":"ContainerStarted","Data":"b6a11c139c712a175109624dcad54dd60ee506cc2e5526da540e53ab5f945bf2"}
Jul 15 11:38:38.703071 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.703023    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4" podStartSLOduration=0.703005682 podStartE2EDuration="703.005682ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:38:38.701494417 +0000 UTC m=+3196.500967942" watchObservedRunningTime="2026-07-15 11:38:38.703005682 +0000 UTC m=+3196.502479207"
Jul 15 11:38:38.707998 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.707974    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e6d07f2-848a-4396-accf-7d65227f0073-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:38.708115 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.708027    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2e6d07f2-848a-4396-accf-7d65227f0073-tokenizer-uds\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:38.708115 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.708056    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlbf2\" (UniqueName: \"kubernetes.io/projected/2e6d07f2-848a-4396-accf-7d65227f0073-kube-api-access-zlbf2\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:38.809072 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.809039    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e6d07f2-848a-4396-accf-7d65227f0073-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:38.809303 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.809085    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2e6d07f2-848a-4396-accf-7d65227f0073-tokenizer-uds\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:38.809303 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.809111    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlbf2\" (UniqueName: \"kubernetes.io/projected/2e6d07f2-848a-4396-accf-7d65227f0073-kube-api-access-zlbf2\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:38.809679 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.809653    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2e6d07f2-848a-4396-accf-7d65227f0073-tokenizer-uds\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:38.811843 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.811820    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e6d07f2-848a-4396-accf-7d65227f0073-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:38.817925 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.817898    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlbf2\" (UniqueName: \"kubernetes.io/projected/2e6d07f2-848a-4396-accf-7d65227f0073-kube-api-access-zlbf2\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") " pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:38.848903 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.848881    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:38.979575 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:38.979548    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"]
Jul 15 11:38:38.981890 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:38:38.981862    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e6d07f2_848a_4396_accf_7d65227f0073.slice/crio-4ef84fde6b9ae27d4d2a9e8a944aaf39228bdf04db410b35abf300315598bc04 WatchSource:0}: Error finding container 4ef84fde6b9ae27d4d2a9e8a944aaf39228bdf04db410b35abf300315598bc04: Status 404 returned error can't find the container with id 4ef84fde6b9ae27d4d2a9e8a944aaf39228bdf04db410b35abf300315598bc04
Jul 15 11:38:39.438502 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:39.438476    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:39.679614 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:39.679575    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6" event={"ID":"2e6d07f2-848a-4396-accf-7d65227f0073","Type":"ContainerStarted","Data":"156d75f63d82a17a2185950bd88878d70f19512d466d1d05751746a798102617"}
Jul 15 11:38:39.679754 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:39.679635    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6" event={"ID":"2e6d07f2-848a-4396-accf-7d65227f0073","Type":"ContainerStarted","Data":"4ef84fde6b9ae27d4d2a9e8a944aaf39228bdf04db410b35abf300315598bc04"}
Jul 15 11:38:39.679855 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:39.679839    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:38:39.698232 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:39.698148    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6" podStartSLOduration=1.698134782 podStartE2EDuration="1.698134782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:38:39.696414888 +0000 UTC m=+3197.495888413" watchObservedRunningTime="2026-07-15 11:38:39.698134782 +0000 UTC m=+3197.497608306"
Jul 15 11:38:48.439153 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:48.439120    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:48.451385 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:48.451366    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:38:48.737238 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:38:48.737211    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:39:10.685656 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:39:10.685630    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:40:43.174016 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:43.173930    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"]
Jul 15 11:40:43.175868 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:43.175839    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"]
Jul 15 11:40:50.685691 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:50.685655    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6" podUID="2e6d07f2-848a-4396-accf-7d65227f0073" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:40:58.203117 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.203056    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4" podUID="36a4a645-3220-4c38-9259-02a9eff0c6d5" containerName="main" containerID="cri-o://49fec28b5de106e809aeb291bdef7d6e5a51de5d609b04d7b3429701d74f5104" gracePeriod=45
Jul 15 11:40:58.205638 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.205605    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6" podUID="2e6d07f2-848a-4396-accf-7d65227f0073" containerName="main" containerID="cri-o://156d75f63d82a17a2185950bd88878d70f19512d466d1d05751746a798102617" gracePeriod=45
Jul 15 11:40:58.324722 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.324694    2536 generic.go:356] "Generic (PLEG): container finished" podID="2e6d07f2-848a-4396-accf-7d65227f0073" containerID="156d75f63d82a17a2185950bd88878d70f19512d466d1d05751746a798102617" exitCode=0
Jul 15 11:40:58.324812 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.324767    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6" event={"ID":"2e6d07f2-848a-4396-accf-7d65227f0073","Type":"ContainerDied","Data":"156d75f63d82a17a2185950bd88878d70f19512d466d1d05751746a798102617"}
Jul 15 11:40:58.326520 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.326495    2536 generic.go:356] "Generic (PLEG): container finished" podID="36a4a645-3220-4c38-9259-02a9eff0c6d5" containerID="49fec28b5de106e809aeb291bdef7d6e5a51de5d609b04d7b3429701d74f5104" exitCode=0
Jul 15 11:40:58.326610 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.326557    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4" event={"ID":"36a4a645-3220-4c38-9259-02a9eff0c6d5","Type":"ContainerDied","Data":"49fec28b5de106e809aeb291bdef7d6e5a51de5d609b04d7b3429701d74f5104"}
Jul 15 11:40:58.477802 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.477781    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:40:58.480430 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.480410    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:40:58.549136 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549101    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-home\") pod \"36a4a645-3220-4c38-9259-02a9eff0c6d5\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") "
Jul 15 11:40:58.549290 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549150    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-model-cache\") pod \"36a4a645-3220-4c38-9259-02a9eff0c6d5\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") "
Jul 15 11:40:58.549290 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549177    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2e6d07f2-848a-4396-accf-7d65227f0073-tokenizer-uds\") pod \"2e6d07f2-848a-4396-accf-7d65227f0073\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") "
Jul 15 11:40:58.549290 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549209    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqzxh\" (UniqueName: \"kubernetes.io/projected/36a4a645-3220-4c38-9259-02a9eff0c6d5-kube-api-access-qqzxh\") pod \"36a4a645-3220-4c38-9259-02a9eff0c6d5\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") "
Jul 15 11:40:58.549290 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549253    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e6d07f2-848a-4396-accf-7d65227f0073-tls-certs\") pod \"2e6d07f2-848a-4396-accf-7d65227f0073\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") "
Jul 15 11:40:58.549585 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549337    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36a4a645-3220-4c38-9259-02a9eff0c6d5-tls-certs\") pod \"36a4a645-3220-4c38-9259-02a9eff0c6d5\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") "
Jul 15 11:40:58.549585 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549393    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-tmp-dir\") pod \"36a4a645-3220-4c38-9259-02a9eff0c6d5\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") "
Jul 15 11:40:58.549585 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549427    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-model-cache" (OuterVolumeSpecName: "model-cache") pod "36a4a645-3220-4c38-9259-02a9eff0c6d5" (UID: "36a4a645-3220-4c38-9259-02a9eff0c6d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:40:58.549585 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549445    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-dshm\") pod \"36a4a645-3220-4c38-9259-02a9eff0c6d5\" (UID: \"36a4a645-3220-4c38-9259-02a9eff0c6d5\") "
Jul 15 11:40:58.549585 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549485    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e6d07f2-848a-4396-accf-7d65227f0073-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "2e6d07f2-848a-4396-accf-7d65227f0073" (UID: "2e6d07f2-848a-4396-accf-7d65227f0073"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:40:58.549585 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549518    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlbf2\" (UniqueName: \"kubernetes.io/projected/2e6d07f2-848a-4396-accf-7d65227f0073-kube-api-access-zlbf2\") pod \"2e6d07f2-848a-4396-accf-7d65227f0073\" (UID: \"2e6d07f2-848a-4396-accf-7d65227f0073\") "
Jul 15 11:40:58.549921 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549666    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-home" (OuterVolumeSpecName: "home") pod "36a4a645-3220-4c38-9259-02a9eff0c6d5" (UID: "36a4a645-3220-4c38-9259-02a9eff0c6d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:40:58.549921 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549856    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:40:58.549921 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549877    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:40:58.549921 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.549892    2536 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/2e6d07f2-848a-4396-accf-7d65227f0073-tokenizer-uds\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:40:58.550119 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.550037    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36a4a645-3220-4c38-9259-02a9eff0c6d5" (UID: "36a4a645-3220-4c38-9259-02a9eff0c6d5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:40:58.551725 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.551700    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36a4a645-3220-4c38-9259-02a9eff0c6d5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36a4a645-3220-4c38-9259-02a9eff0c6d5" (UID: "36a4a645-3220-4c38-9259-02a9eff0c6d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:40:58.551985 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.551947    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-dshm" (OuterVolumeSpecName: "dshm") pod "36a4a645-3220-4c38-9259-02a9eff0c6d5" (UID: "36a4a645-3220-4c38-9259-02a9eff0c6d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:40:58.552217 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.552179    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36a4a645-3220-4c38-9259-02a9eff0c6d5-kube-api-access-qqzxh" (OuterVolumeSpecName: "kube-api-access-qqzxh") pod "36a4a645-3220-4c38-9259-02a9eff0c6d5" (UID: "36a4a645-3220-4c38-9259-02a9eff0c6d5"). InnerVolumeSpecName "kube-api-access-qqzxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:40:58.552350 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.552226    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e6d07f2-848a-4396-accf-7d65227f0073-kube-api-access-zlbf2" (OuterVolumeSpecName: "kube-api-access-zlbf2") pod "2e6d07f2-848a-4396-accf-7d65227f0073" (UID: "2e6d07f2-848a-4396-accf-7d65227f0073"). InnerVolumeSpecName "kube-api-access-zlbf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:40:58.552350 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.552244    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e6d07f2-848a-4396-accf-7d65227f0073-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2e6d07f2-848a-4396-accf-7d65227f0073" (UID: "2e6d07f2-848a-4396-accf-7d65227f0073"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:40:58.650918 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.650893    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:40:58.650918 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.650918    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlbf2\" (UniqueName: \"kubernetes.io/projected/2e6d07f2-848a-4396-accf-7d65227f0073-kube-api-access-zlbf2\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:40:58.651071 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.650929    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqzxh\" (UniqueName: \"kubernetes.io/projected/36a4a645-3220-4c38-9259-02a9eff0c6d5-kube-api-access-qqzxh\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:40:58.651071 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.650938    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e6d07f2-848a-4396-accf-7d65227f0073-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:40:58.651071 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.650947    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36a4a645-3220-4c38-9259-02a9eff0c6d5-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:40:58.651071 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:58.650974    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36a4a645-3220-4c38-9259-02a9eff0c6d5-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:40:59.332600 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:59.332571    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"
Jul 15 11:40:59.333013 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:59.332571    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6" event={"ID":"2e6d07f2-848a-4396-accf-7d65227f0073","Type":"ContainerDied","Data":"4ef84fde6b9ae27d4d2a9e8a944aaf39228bdf04db410b35abf300315598bc04"}
Jul 15 11:40:59.333013 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:59.332707    2536 scope.go:117] "RemoveContainer" containerID="156d75f63d82a17a2185950bd88878d70f19512d466d1d05751746a798102617"
Jul 15 11:40:59.334109 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:59.333937    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4" event={"ID":"36a4a645-3220-4c38-9259-02a9eff0c6d5","Type":"ContainerDied","Data":"b6a11c139c712a175109624dcad54dd60ee506cc2e5526da540e53ab5f945bf2"}
Jul 15 11:40:59.334109 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:59.333948    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"
Jul 15 11:40:59.341449 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:59.341431    2536 scope.go:117] "RemoveContainer" containerID="49fec28b5de106e809aeb291bdef7d6e5a51de5d609b04d7b3429701d74f5104"
Jul 15 11:40:59.351317 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:59.351297    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"]
Jul 15 11:40:59.355527 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:59.355497    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schek7mx6"]
Jul 15 11:40:59.366632 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:59.366612    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"]
Jul 15 11:40:59.370985 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:40:59.370960    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4pxls4"]
Jul 15 11:41:00.594959 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:00.594923    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e6d07f2-848a-4396-accf-7d65227f0073" path="/var/lib/kubelet/pods/2e6d07f2-848a-4396-accf-7d65227f0073/volumes"
Jul 15 11:41:00.595348 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:00.595302    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36a4a645-3220-4c38-9259-02a9eff0c6d5" path="/var/lib/kubelet/pods/36a4a645-3220-4c38-9259-02a9eff0c6d5/volumes"
Jul 15 11:41:24.316688 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.316657    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"]
Jul 15 11:41:24.317093 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.316999    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e6d07f2-848a-4396-accf-7d65227f0073" containerName="main"
Jul 15 11:41:24.317093 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.317012    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e6d07f2-848a-4396-accf-7d65227f0073" containerName="main"
Jul 15 11:41:24.317093 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.317023    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a4a645-3220-4c38-9259-02a9eff0c6d5" containerName="main"
Jul 15 11:41:24.317093 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.317028    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a4a645-3220-4c38-9259-02a9eff0c6d5" containerName="main"
Jul 15 11:41:24.317093 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.317085    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a4a645-3220-4c38-9259-02a9eff0c6d5" containerName="main"
Jul 15 11:41:24.317093 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.317093    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e6d07f2-848a-4396-accf-7d65227f0073" containerName="main"
Jul 15 11:41:24.321431 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.321414    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.324673 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.324649    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:41:24.324673 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.324663    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:41:24.324848 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.324682    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:41:24.324848 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.324665    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Jul 15 11:41:24.333212 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.333185    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"]
Jul 15 11:41:24.463374 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.463347    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsns6\" (UniqueName: \"kubernetes.io/projected/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-kube-api-access-lsns6\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.463505 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.463385    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.463505 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.463419    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-model-cache\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.463505 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.463479    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-dshm\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.463632 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.463504    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.463632 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.463557    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-home\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.564349 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.564325    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.564480 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.564359    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-home\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.564480 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.564411    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsns6\" (UniqueName: \"kubernetes.io/projected/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-kube-api-access-lsns6\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.564613 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.564594    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.564690 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.564675    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-model-cache\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.564747 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.564716    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-dshm\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.564816 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.564793    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-home\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.564932 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.564912    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.564995 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.564976    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-model-cache\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.567003 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.566943    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-dshm\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.567003 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.566945    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.573391 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.573370    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsns6\" (UniqueName: \"kubernetes.io/projected/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-kube-api-access-lsns6\") pod \"scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.631748 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.631726    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:24.769929 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:24.769899    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"]
Jul 15 11:41:24.773228 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:41:24.773206    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f21fcf8_dce7_4c3e_975c_c4a4249fc892.slice/crio-d7c0fe852b97156a7d1debd560a08cc1fb68e5c3c828ec6c61de58556d719b6e WatchSource:0}: Error finding container d7c0fe852b97156a7d1debd560a08cc1fb68e5c3c828ec6c61de58556d719b6e: Status 404 returned error can't find the container with id d7c0fe852b97156a7d1debd560a08cc1fb68e5c3c828ec6c61de58556d719b6e
Jul 15 11:41:25.453188 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:25.453153    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x" event={"ID":"9f21fcf8-dce7-4c3e-975c-c4a4249fc892","Type":"ContainerStarted","Data":"97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6"}
Jul 15 11:41:25.453690 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:25.453214    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x" event={"ID":"9f21fcf8-dce7-4c3e-975c-c4a4249fc892","Type":"ContainerStarted","Data":"d7c0fe852b97156a7d1debd560a08cc1fb68e5c3c828ec6c61de58556d719b6e"}
Jul 15 11:41:25.478777 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:25.478731    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x" podStartSLOduration=1.4787169 podStartE2EDuration="1.4787169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:41:25.477320579 +0000 UTC m=+3363.276794104" watchObservedRunningTime="2026-07-15 11:41:25.4787169 +0000 UTC m=+3363.278190425"
Jul 15 11:41:25.632579 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:25.632548    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:34.632181 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:34.632149    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:34.644692 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:34.644669    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:35.507751 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:35.507725    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:41:59.108403 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:41:59.108372    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"]
Jul 15 11:42:14.129418 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.129316    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x" podUID="9f21fcf8-dce7-4c3e-975c-c4a4249fc892" containerName="main" containerID="cri-o://97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6" gracePeriod=45
Jul 15 11:42:14.384011 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.383960    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:42:14.477369 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.477337    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tls-certs\") pod \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") "
Jul 15 11:42:14.477522 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.477382    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tmp-dir\") pod \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") "
Jul 15 11:42:14.477522 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.477435    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-model-cache\") pod \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") "
Jul 15 11:42:14.477522 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.477460    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsns6\" (UniqueName: \"kubernetes.io/projected/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-kube-api-access-lsns6\") pod \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") "
Jul 15 11:42:14.477522 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.477501    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-dshm\") pod \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") "
Jul 15 11:42:14.477522 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.477520    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-home\") pod \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\" (UID: \"9f21fcf8-dce7-4c3e-975c-c4a4249fc892\") "
Jul 15 11:42:14.477777 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.477681    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f21fcf8-dce7-4c3e-975c-c4a4249fc892" (UID: "9f21fcf8-dce7-4c3e-975c-c4a4249fc892"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:42:14.477777 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.477694    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f21fcf8-dce7-4c3e-975c-c4a4249fc892" (UID: "9f21fcf8-dce7-4c3e-975c-c4a4249fc892"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:42:14.477878 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.477858    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:42:14.477921 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.477862    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-home" (OuterVolumeSpecName: "home") pod "9f21fcf8-dce7-4c3e-975c-c4a4249fc892" (UID: "9f21fcf8-dce7-4c3e-975c-c4a4249fc892"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:42:14.477921 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.477884    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:42:14.479579 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.479553    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f21fcf8-dce7-4c3e-975c-c4a4249fc892" (UID: "9f21fcf8-dce7-4c3e-975c-c4a4249fc892"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:42:14.479734 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.479718    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-kube-api-access-lsns6" (OuterVolumeSpecName: "kube-api-access-lsns6") pod "9f21fcf8-dce7-4c3e-975c-c4a4249fc892" (UID: "9f21fcf8-dce7-4c3e-975c-c4a4249fc892"). InnerVolumeSpecName "kube-api-access-lsns6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:42:14.479795 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.479744    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-dshm" (OuterVolumeSpecName: "dshm") pod "9f21fcf8-dce7-4c3e-975c-c4a4249fc892" (UID: "9f21fcf8-dce7-4c3e-975c-c4a4249fc892"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:42:14.578980 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.578954    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsns6\" (UniqueName: \"kubernetes.io/projected/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-kube-api-access-lsns6\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:42:14.578980 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.578978    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:42:14.578980 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.578987    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:42:14.579168 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.578995    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f21fcf8-dce7-4c3e-975c-c4a4249fc892-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:42:14.679903 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.679834    2536 generic.go:356] "Generic (PLEG): container finished" podID="9f21fcf8-dce7-4c3e-975c-c4a4249fc892" containerID="97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6" exitCode=0
Jul 15 11:42:14.679903 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.679884    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"
Jul 15 11:42:14.680082 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.679915    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x" event={"ID":"9f21fcf8-dce7-4c3e-975c-c4a4249fc892","Type":"ContainerDied","Data":"97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6"}
Jul 15 11:42:14.680082 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.679967    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x" event={"ID":"9f21fcf8-dce7-4c3e-975c-c4a4249fc892","Type":"ContainerDied","Data":"d7c0fe852b97156a7d1debd560a08cc1fb68e5c3c828ec6c61de58556d719b6e"}
Jul 15 11:42:14.680082 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.679988    2536 scope.go:117] "RemoveContainer" containerID="97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6"
Jul 15 11:42:14.688196 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.688180    2536 scope.go:117] "RemoveContainer" containerID="97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6"
Jul 15 11:42:14.688470 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:42:14.688450    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6\": container with ID starting with 97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6 not found: ID does not exist" containerID="97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6"
Jul 15 11:42:14.688525 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.688478    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6"} err="failed to get container status \"97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6\": rpc error: code = NotFound desc = could not find container \"97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6\": container with ID starting with 97b298fead5ce1784ee0e17d0d9a74edba1b624432471e4f02b7ef9c033a9ec6 not found: ID does not exist"
Jul 15 11:42:14.695683 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.695665    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"]
Jul 15 11:42:14.701453 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:14.701431    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-fmp2x"]
Jul 15 11:42:16.595114 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:16.595080    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f21fcf8-dce7-4c3e-975c-c4a4249fc892" path="/var/lib/kubelet/pods/9f21fcf8-dce7-4c3e-975c-c4a4249fc892/volumes"
Jul 15 11:42:31.112473 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.112440    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"]
Jul 15 11:42:31.113038 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.112941    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f21fcf8-dce7-4c3e-975c-c4a4249fc892" containerName="main"
Jul 15 11:42:31.113038 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.112960    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f21fcf8-dce7-4c3e-975c-c4a4249fc892" containerName="main"
Jul 15 11:42:31.113160 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.113044    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f21fcf8-dce7-4c3e-975c-c4a4249fc892" containerName="main"
Jul 15 11:42:31.117479 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.117458    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.120726 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.120708    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:42:31.120853 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.120707    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:42:31.120905 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.120880    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Jul 15 11:42:31.120955 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.120896    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:42:31.125748 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.125719    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"]
Jul 15 11:42:31.210499 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.210465    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm7x4\" (UniqueName: \"kubernetes.io/projected/71d65097-04fc-43c8-ab06-3e1744ee5496-kube-api-access-qm7x4\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.210660 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.210516    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-model-cache\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.210660 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.210543    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-home\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.210660 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.210572    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71d65097-04fc-43c8-ab06-3e1744ee5496-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.210660 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.210620    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-dshm\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.210660 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.210647    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.311665 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.311640    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-dshm\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.311832 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.311678    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.311832 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.311751    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm7x4\" (UniqueName: \"kubernetes.io/projected/71d65097-04fc-43c8-ab06-3e1744ee5496-kube-api-access-qm7x4\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.311832 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.311790    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-model-cache\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.311990 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.311901    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-home\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.311990 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.311939    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71d65097-04fc-43c8-ab06-3e1744ee5496-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.312125 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.312107    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.312202 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.312182    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-model-cache\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.312253 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.312215    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-home\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.314111 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.314089    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-dshm\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.314444 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.314429    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71d65097-04fc-43c8-ab06-3e1744ee5496-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.321458 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.321434    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm7x4\" (UniqueName: \"kubernetes.io/projected/71d65097-04fc-43c8-ab06-3e1744ee5496-kube-api-access-qm7x4\") pod \"scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.429824 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.429767    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:31.537945 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.537910    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"]
Jul 15 11:42:31.542622 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.542601    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:31.544943 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.544925    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-6qmnm\""
Jul 15 11:42:31.550931 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.550694    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"]
Jul 15 11:42:31.577013 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.576976    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"]
Jul 15 11:42:31.582971 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:42:31.582924    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71d65097_04fc_43c8_ab06_3e1744ee5496.slice/crio-f8c1ac3c48330307f7bb209568c8f744a0ba274371d16988f6ff6e4d4bd2f300 WatchSource:0}: Error finding container f8c1ac3c48330307f7bb209568c8f744a0ba274371d16988f6ff6e4d4bd2f300: Status 404 returned error can't find the container with id f8c1ac3c48330307f7bb209568c8f744a0ba274371d16988f6ff6e4d4bd2f300
Jul 15 11:42:31.715987 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.715958    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7s88\" (UniqueName: \"kubernetes.io/projected/33980c8d-1d97-4626-a666-bba39ff9c36e-kube-api-access-f7s88\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:31.716122 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.716003    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33980c8d-1d97-4626-a666-bba39ff9c36e-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:31.716122 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.716085    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/33980c8d-1d97-4626-a666-bba39ff9c36e-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:31.756616 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.756581    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf" event={"ID":"71d65097-04fc-43c8-ab06-3e1744ee5496","Type":"ContainerStarted","Data":"e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1"}
Jul 15 11:42:31.756755 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.756643    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf" event={"ID":"71d65097-04fc-43c8-ab06-3e1744ee5496","Type":"ContainerStarted","Data":"f8c1ac3c48330307f7bb209568c8f744a0ba274371d16988f6ff6e4d4bd2f300"}
Jul 15 11:42:31.775154 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.775106    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf" podStartSLOduration=0.775087004 podStartE2EDuration="775.087004ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:42:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:42:31.774456419 +0000 UTC m=+3429.573929947" watchObservedRunningTime="2026-07-15 11:42:31.775087004 +0000 UTC m=+3429.574560529"
Jul 15 11:42:31.816835 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.816807    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/33980c8d-1d97-4626-a666-bba39ff9c36e-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:31.816937 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.816887    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7s88\" (UniqueName: \"kubernetes.io/projected/33980c8d-1d97-4626-a666-bba39ff9c36e-kube-api-access-f7s88\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:31.816937 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.816917    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33980c8d-1d97-4626-a666-bba39ff9c36e-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:31.817172 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.817154    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/33980c8d-1d97-4626-a666-bba39ff9c36e-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:31.819424 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.819397    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33980c8d-1d97-4626-a666-bba39ff9c36e-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:31.824929 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.824906    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7s88\" (UniqueName: \"kubernetes.io/projected/33980c8d-1d97-4626-a666-bba39ff9c36e-kube-api-access-f7s88\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:31.853953 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.853929    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:31.990324 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:31.990221    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"]
Jul 15 11:42:31.993796 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:42:31.993775    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33980c8d_1d97_4626_a666_bba39ff9c36e.slice/crio-943605cb27c475c902d3e9e4959b5e5d7198c9a980eaa11dcf61e2637dbe1799 WatchSource:0}: Error finding container 943605cb27c475c902d3e9e4959b5e5d7198c9a980eaa11dcf61e2637dbe1799: Status 404 returned error can't find the container with id 943605cb27c475c902d3e9e4959b5e5d7198c9a980eaa11dcf61e2637dbe1799
Jul 15 11:42:32.430135 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:32.430063    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:32.762118 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:32.762082    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" event={"ID":"33980c8d-1d97-4626-a666-bba39ff9c36e","Type":"ContainerStarted","Data":"8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69"}
Jul 15 11:42:32.762324 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:32.762142    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" event={"ID":"33980c8d-1d97-4626-a666-bba39ff9c36e","Type":"ContainerStarted","Data":"943605cb27c475c902d3e9e4959b5e5d7198c9a980eaa11dcf61e2637dbe1799"}
Jul 15 11:42:32.781474 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:32.781428    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podStartSLOduration=1.78141241 podStartE2EDuration="1.78141241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:42:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:42:32.78035579 +0000 UTC m=+3430.579829319" watchObservedRunningTime="2026-07-15 11:42:32.78141241 +0000 UTC m=+3430.580885974"
Jul 15 11:42:33.767564 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:33.767538    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:42:41.430910 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:41.430872    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:41.443176 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:41.443152    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:42:41.814685 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:42:41.814659    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:43:04.774079 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:43:04.774042    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:43:14.773953 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:43:14.773916    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:43:24.774545 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:43:24.774495    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:43:34.774418 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:43:34.774375    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:43:42.592621 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:43:42.592575    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:43:52.592848 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:43:52.592795    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:44:02.592854 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:02.592823    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:44:12.592912 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:12.592869    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:44:22.592808 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:22.592767    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:44:25.779014 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:25.778980    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"]
Jul 15 11:44:25.780978 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:25.780945    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:44:25.782858 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:25.782835    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"]
Jul 15 11:44:35.781115 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:35.781076    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:44:40.799565 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:40.799517    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main" containerID="cri-o://8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69" gracePeriod=45
Jul 15 11:44:40.802724 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:40.802686    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf" podUID="71d65097-04fc-43c8-ab06-3e1744ee5496" containerName="main" containerID="cri-o://e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1" gracePeriod=45
Jul 15 11:44:40.807430 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:40.807401    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf" podUID="71d65097-04fc-43c8-ab06-3e1744ee5496" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/health\": EOF"
Jul 15 11:44:41.065112 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.065090    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:44:41.068025 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.068006    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:44:41.098153 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098122    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-tmp-dir\") pod \"71d65097-04fc-43c8-ab06-3e1744ee5496\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") "
Jul 15 11:44:41.098153 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098151    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7s88\" (UniqueName: \"kubernetes.io/projected/33980c8d-1d97-4626-a666-bba39ff9c36e-kube-api-access-f7s88\") pod \"33980c8d-1d97-4626-a666-bba39ff9c36e\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") "
Jul 15 11:44:41.098382 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098173    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-dshm\") pod \"71d65097-04fc-43c8-ab06-3e1744ee5496\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") "
Jul 15 11:44:41.098382 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098198    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-model-cache\") pod \"71d65097-04fc-43c8-ab06-3e1744ee5496\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") "
Jul 15 11:44:41.098382 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098227    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71d65097-04fc-43c8-ab06-3e1744ee5496-tls-certs\") pod \"71d65097-04fc-43c8-ab06-3e1744ee5496\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") "
Jul 15 11:44:41.098382 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098293    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33980c8d-1d97-4626-a666-bba39ff9c36e-tls-certs\") pod \"33980c8d-1d97-4626-a666-bba39ff9c36e\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") "
Jul 15 11:44:41.098382 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098337    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm7x4\" (UniqueName: \"kubernetes.io/projected/71d65097-04fc-43c8-ab06-3e1744ee5496-kube-api-access-qm7x4\") pod \"71d65097-04fc-43c8-ab06-3e1744ee5496\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") "
Jul 15 11:44:41.098382 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098365    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-home\") pod \"71d65097-04fc-43c8-ab06-3e1744ee5496\" (UID: \"71d65097-04fc-43c8-ab06-3e1744ee5496\") "
Jul 15 11:44:41.098711 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098391    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/33980c8d-1d97-4626-a666-bba39ff9c36e-tokenizer-uds\") pod \"33980c8d-1d97-4626-a666-bba39ff9c36e\" (UID: \"33980c8d-1d97-4626-a666-bba39ff9c36e\") "
Jul 15 11:44:41.098711 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098574    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "71d65097-04fc-43c8-ab06-3e1744ee5496" (UID: "71d65097-04fc-43c8-ab06-3e1744ee5496"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:44:41.098826 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098728    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:44:41.098826 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098760    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33980c8d-1d97-4626-a666-bba39ff9c36e-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "33980c8d-1d97-4626-a666-bba39ff9c36e" (UID: "33980c8d-1d97-4626-a666-bba39ff9c36e"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:44:41.098826 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.098760    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-model-cache" (OuterVolumeSpecName: "model-cache") pod "71d65097-04fc-43c8-ab06-3e1744ee5496" (UID: "71d65097-04fc-43c8-ab06-3e1744ee5496"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:44:41.099136 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.099109    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-home" (OuterVolumeSpecName: "home") pod "71d65097-04fc-43c8-ab06-3e1744ee5496" (UID: "71d65097-04fc-43c8-ab06-3e1744ee5496"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:44:41.101102 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.101065    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33980c8d-1d97-4626-a666-bba39ff9c36e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "33980c8d-1d97-4626-a666-bba39ff9c36e" (UID: "33980c8d-1d97-4626-a666-bba39ff9c36e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:44:41.101102 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.101087    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33980c8d-1d97-4626-a666-bba39ff9c36e-kube-api-access-f7s88" (OuterVolumeSpecName: "kube-api-access-f7s88") pod "33980c8d-1d97-4626-a666-bba39ff9c36e" (UID: "33980c8d-1d97-4626-a666-bba39ff9c36e"). InnerVolumeSpecName "kube-api-access-f7s88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:44:41.101312 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.101230    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71d65097-04fc-43c8-ab06-3e1744ee5496-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "71d65097-04fc-43c8-ab06-3e1744ee5496" (UID: "71d65097-04fc-43c8-ab06-3e1744ee5496"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:44:41.101312 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.101240    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d65097-04fc-43c8-ab06-3e1744ee5496-kube-api-access-qm7x4" (OuterVolumeSpecName: "kube-api-access-qm7x4") pod "71d65097-04fc-43c8-ab06-3e1744ee5496" (UID: "71d65097-04fc-43c8-ab06-3e1744ee5496"). InnerVolumeSpecName "kube-api-access-qm7x4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:44:41.101520 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.101494    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-dshm" (OuterVolumeSpecName: "dshm") pod "71d65097-04fc-43c8-ab06-3e1744ee5496" (UID: "71d65097-04fc-43c8-ab06-3e1744ee5496"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:44:41.199984 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.199955    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71d65097-04fc-43c8-ab06-3e1744ee5496-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:44:41.199984 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.199977    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33980c8d-1d97-4626-a666-bba39ff9c36e-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:44:41.199984 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.199986    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm7x4\" (UniqueName: \"kubernetes.io/projected/71d65097-04fc-43c8-ab06-3e1744ee5496-kube-api-access-qm7x4\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:44:41.200191 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.199997    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:44:41.200191 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.200005    2536 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/33980c8d-1d97-4626-a666-bba39ff9c36e-tokenizer-uds\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:44:41.200191 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.200015    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7s88\" (UniqueName: \"kubernetes.io/projected/33980c8d-1d97-4626-a666-bba39ff9c36e-kube-api-access-f7s88\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:44:41.200191 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.200022    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:44:41.200191 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.200030    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71d65097-04fc-43c8-ab06-3e1744ee5496-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:44:41.342231 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.342145    2536 generic.go:356] "Generic (PLEG): container finished" podID="71d65097-04fc-43c8-ab06-3e1744ee5496" containerID="e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1" exitCode=0
Jul 15 11:44:41.342231 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.342204    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"
Jul 15 11:44:41.342464 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.342236    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf" event={"ID":"71d65097-04fc-43c8-ab06-3e1744ee5496","Type":"ContainerDied","Data":"e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1"}
Jul 15 11:44:41.342464 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.342315    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf" event={"ID":"71d65097-04fc-43c8-ab06-3e1744ee5496","Type":"ContainerDied","Data":"f8c1ac3c48330307f7bb209568c8f744a0ba274371d16988f6ff6e4d4bd2f300"}
Jul 15 11:44:41.342464 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.342350    2536 scope.go:117] "RemoveContainer" containerID="e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1"
Jul 15 11:44:41.343622 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.343600    2536 generic.go:356] "Generic (PLEG): container finished" podID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerID="8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69" exitCode=0
Jul 15 11:44:41.343711 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.343639    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"
Jul 15 11:44:41.343711 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.343684    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" event={"ID":"33980c8d-1d97-4626-a666-bba39ff9c36e","Type":"ContainerDied","Data":"8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69"}
Jul 15 11:44:41.343808 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.343729    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m" event={"ID":"33980c8d-1d97-4626-a666-bba39ff9c36e","Type":"ContainerDied","Data":"943605cb27c475c902d3e9e4959b5e5d7198c9a980eaa11dcf61e2637dbe1799"}
Jul 15 11:44:41.351186 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.351161    2536 scope.go:117] "RemoveContainer" containerID="e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1"
Jul 15 11:44:41.351484 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:44:41.351465    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1\": container with ID starting with e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1 not found: ID does not exist" containerID="e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1"
Jul 15 11:44:41.351574 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.351492    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1"} err="failed to get container status \"e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1\": rpc error: code = NotFound desc = could not find container \"e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1\": container with ID starting with e48fb500f5356d85bbeffd5e42b500e8527ae1a3cc53fc6639be6a25903737c1 not found: ID does not exist"
Jul 15 11:44:41.351574 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.351509    2536 scope.go:117] "RemoveContainer" containerID="8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69"
Jul 15 11:44:41.359478 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.359463    2536 scope.go:117] "RemoveContainer" containerID="8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69"
Jul 15 11:44:41.359714 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:44:41.359697    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69\": container with ID starting with 8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69 not found: ID does not exist" containerID="8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69"
Jul 15 11:44:41.359765 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.359722    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69"} err="failed to get container status \"8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69\": rpc error: code = NotFound desc = could not find container \"8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69\": container with ID starting with 8da3c2e907acca9eb3be06a4da9448e6e30453bdad216b62223c258da0256a69 not found: ID does not exist"
Jul 15 11:44:41.371537 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.371514    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"]
Jul 15 11:44:41.379767 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.379750    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-d5vqf"]
Jul 15 11:44:41.404383 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.404362    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"]
Jul 15 11:44:41.409286 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:41.409259    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-85cd85fprk2m"]
Jul 15 11:44:42.594861 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:42.594829    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" path="/var/lib/kubelet/pods/33980c8d-1d97-4626-a666-bba39ff9c36e/volumes"
Jul 15 11:44:42.595376 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:44:42.595359    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71d65097-04fc-43c8-ab06-3e1744ee5496" path="/var/lib/kubelet/pods/71d65097-04fc-43c8-ab06-3e1744ee5496/volumes"
Jul 15 11:45:00.269596 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.269564    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"]
Jul 15 11:45:00.270155 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.270076    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71d65097-04fc-43c8-ab06-3e1744ee5496" containerName="main"
Jul 15 11:45:00.270155 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.270096    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d65097-04fc-43c8-ab06-3e1744ee5496" containerName="main"
Jul 15 11:45:00.270155 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.270107    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main"
Jul 15 11:45:00.270155 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.270115    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main"
Jul 15 11:45:00.270410 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.270192    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="71d65097-04fc-43c8-ab06-3e1744ee5496" containerName="main"
Jul 15 11:45:00.270410 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.270212    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="33980c8d-1d97-4626-a666-bba39ff9c36e" containerName="main"
Jul 15 11:45:00.274616 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.274595    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.277257 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.277235    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:45:00.278238 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.278221    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:45:00.278357 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.278226    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-custom-template-test-epp-sa-dockercfg-k6tx2\""
Jul 15 11:45:00.278426 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.278359    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-custom-template-test-kserve-self-signed-certs\""
Jul 15 11:45:00.278538 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.278522    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:45:00.284262 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.284233    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"]
Jul 15 11:45:00.355190 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.355164    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.355359 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.355201    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a517b215-9009-4a61-a3d7-4e08fefaa934-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.355359 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.355339    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6bbf\" (UniqueName: \"kubernetes.io/projected/a517b215-9009-4a61-a3d7-4e08fefaa934-kube-api-access-g6bbf\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.355437 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.355382    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.355437 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.355398    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.455847 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.455818    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6bbf\" (UniqueName: \"kubernetes.io/projected/a517b215-9009-4a61-a3d7-4e08fefaa934-kube-api-access-g6bbf\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.455985 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.455859    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.455985 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.455877    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.455985 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.455912    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.455985 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.455935    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a517b215-9009-4a61-a3d7-4e08fefaa934-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.456265 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.456246    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.456337 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.456267    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-uds\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.456373 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.456353    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.458648 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.458628    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a517b215-9009-4a61-a3d7-4e08fefaa934-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.463996 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.463973    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6bbf\" (UniqueName: \"kubernetes.io/projected/a517b215-9009-4a61-a3d7-4e08fefaa934-kube-api-access-g6bbf\") pod \"scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") " pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.584649 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.584584    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:00.718688 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:00.718659    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"]
Jul 15 11:45:00.722039 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:45:00.722010    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda517b215_9009_4a61_a3d7_4e08fefaa934.slice/crio-948c82ee0f043fbcbcb5e846f4ced908b4a212cd92b9e8d3de63cfd6c8e0ef15 WatchSource:0}: Error finding container 948c82ee0f043fbcbcb5e846f4ced908b4a212cd92b9e8d3de63cfd6c8e0ef15: Status 404 returned error can't find the container with id 948c82ee0f043fbcbcb5e846f4ced908b4a212cd92b9e8d3de63cfd6c8e0ef15
Jul 15 11:45:01.441601 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:01.441566    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k" event={"ID":"a517b215-9009-4a61-a3d7-4e08fefaa934","Type":"ContainerStarted","Data":"52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6"}
Jul 15 11:45:01.442004 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:01.441616    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k" event={"ID":"a517b215-9009-4a61-a3d7-4e08fefaa934","Type":"ContainerStarted","Data":"948c82ee0f043fbcbcb5e846f4ced908b4a212cd92b9e8d3de63cfd6c8e0ef15"}
Jul 15 11:45:01.442004 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:01.441703    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:01.465326 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:01.465250    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k" podStartSLOduration=1.465231618 podStartE2EDuration="1.465231618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:45:01.462961324 +0000 UTC m=+3579.262434850" watchObservedRunningTime="2026-07-15 11:45:01.465231618 +0000 UTC m=+3579.264705146"
Jul 15 11:45:32.448313 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:32.448260    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:33.815320 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:33.815267    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"]
Jul 15 11:45:42.447842 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:42.447806    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k" podUID="a517b215-9009-4a61-a3d7-4e08fefaa934" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 11:45:48.835462 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:48.835417    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k" podUID="a517b215-9009-4a61-a3d7-4e08fefaa934" containerName="main" containerID="cri-o://52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6" gracePeriod=45
Jul 15 11:45:49.089008 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.088946    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:49.156574 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.156547    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-cachi2-cache\") pod \"a517b215-9009-4a61-a3d7-4e08fefaa934\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") "
Jul 15 11:45:49.156760 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.156598    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a517b215-9009-4a61-a3d7-4e08fefaa934-tls-certs\") pod \"a517b215-9009-4a61-a3d7-4e08fefaa934\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") "
Jul 15 11:45:49.156760 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.156617    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-uds\") pod \"a517b215-9009-4a61-a3d7-4e08fefaa934\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") "
Jul 15 11:45:49.156760 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.156645    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-cache\") pod \"a517b215-9009-4a61-a3d7-4e08fefaa934\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") "
Jul 15 11:45:49.156760 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.156733    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6bbf\" (UniqueName: \"kubernetes.io/projected/a517b215-9009-4a61-a3d7-4e08fefaa934-kube-api-access-g6bbf\") pod \"a517b215-9009-4a61-a3d7-4e08fefaa934\" (UID: \"a517b215-9009-4a61-a3d7-4e08fefaa934\") "
Jul 15 11:45:49.156974 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.156871    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-cachi2-cache" (OuterVolumeSpecName: "cachi2-cache") pod "a517b215-9009-4a61-a3d7-4e08fefaa934" (UID: "a517b215-9009-4a61-a3d7-4e08fefaa934"). InnerVolumeSpecName "cachi2-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:45:49.156974 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.156881    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "a517b215-9009-4a61-a3d7-4e08fefaa934" (UID: "a517b215-9009-4a61-a3d7-4e08fefaa934"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:45:49.156974 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.156955    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "a517b215-9009-4a61-a3d7-4e08fefaa934" (UID: "a517b215-9009-4a61-a3d7-4e08fefaa934"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:45:49.158841 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.158808    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a517b215-9009-4a61-a3d7-4e08fefaa934-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a517b215-9009-4a61-a3d7-4e08fefaa934" (UID: "a517b215-9009-4a61-a3d7-4e08fefaa934"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:45:49.159002 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.158982    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a517b215-9009-4a61-a3d7-4e08fefaa934-kube-api-access-g6bbf" (OuterVolumeSpecName: "kube-api-access-g6bbf") pod "a517b215-9009-4a61-a3d7-4e08fefaa934" (UID: "a517b215-9009-4a61-a3d7-4e08fefaa934"). InnerVolumeSpecName "kube-api-access-g6bbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:45:49.257592 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.257561    2536 reconciler_common.go:299] "Volume detached for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-cachi2-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:45:49.257592 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.257588    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a517b215-9009-4a61-a3d7-4e08fefaa934-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:45:49.257592 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.257598    2536 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-uds\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:45:49.257811 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.257607    2536 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/a517b215-9009-4a61-a3d7-4e08fefaa934-tokenizer-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:45:49.257811 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.257617    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6bbf\" (UniqueName: \"kubernetes.io/projected/a517b215-9009-4a61-a3d7-4e08fefaa934-kube-api-access-g6bbf\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:45:49.666738 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.666708    2536 generic.go:356] "Generic (PLEG): container finished" podID="a517b215-9009-4a61-a3d7-4e08fefaa934" containerID="52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6" exitCode=0
Jul 15 11:45:49.666914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.666743    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"
Jul 15 11:45:49.666914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.666786    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k" event={"ID":"a517b215-9009-4a61-a3d7-4e08fefaa934","Type":"ContainerDied","Data":"52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6"}
Jul 15 11:45:49.666914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.666834    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k" event={"ID":"a517b215-9009-4a61-a3d7-4e08fefaa934","Type":"ContainerDied","Data":"948c82ee0f043fbcbcb5e846f4ced908b4a212cd92b9e8d3de63cfd6c8e0ef15"}
Jul 15 11:45:49.666914 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.666854    2536 scope.go:117] "RemoveContainer" containerID="52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6"
Jul 15 11:45:49.675923 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.675908    2536 scope.go:117] "RemoveContainer" containerID="52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6"
Jul 15 11:45:49.676197 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:45:49.676180    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6\": container with ID starting with 52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6 not found: ID does not exist" containerID="52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6"
Jul 15 11:45:49.676252 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.676204    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6"} err="failed to get container status \"52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6\": rpc error: code = NotFound desc = could not find container \"52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6\": container with ID starting with 52dacc6e0dffa707af5aec997f19c88538ddb88f1185125a98bdd134908048c6 not found: ID does not exist"
Jul 15 11:45:49.687410 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.687382    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"]
Jul 15 11:45:49.693458 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:49.693437    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-89dzlr7k"]
Jul 15 11:45:50.594902 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:50.594858    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a517b215-9009-4a61-a3d7-4e08fefaa934" path="/var/lib/kubelet/pods/a517b215-9009-4a61-a3d7-4e08fefaa934/volumes"
Jul 15 11:45:56.744257 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.744221    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"]
Jul 15 11:45:56.744677 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.744593    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a517b215-9009-4a61-a3d7-4e08fefaa934" containerName="main"
Jul 15 11:45:56.744677 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.744605    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="a517b215-9009-4a61-a3d7-4e08fefaa934" containerName="main"
Jul 15 11:45:56.744677 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.744673    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="a517b215-9009-4a61-a3d7-4e08fefaa934" containerName="main"
Jul 15 11:45:56.747605 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.747585    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.751320 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.751296    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:45:56.751320 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.751314    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:45:56.751486 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.751340    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:45:56.751486 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.751374    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Jul 15 11:45:56.758978 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.758958    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"]
Jul 15 11:45:56.815242 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.815218    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.815366 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.815299    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.815366 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.815325    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.815366 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.815341    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v6xs\" (UniqueName: \"kubernetes.io/projected/748c3069-9b31-49cd-b153-1897c2a687d2-kube-api-access-5v6xs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.815485 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.815369    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.815534 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.815481    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/748c3069-9b31-49cd-b153-1897c2a687d2-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.916475 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.916447    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.916584 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.916498    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.916584 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.916518    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.916584 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.916540    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v6xs\" (UniqueName: \"kubernetes.io/projected/748c3069-9b31-49cd-b153-1897c2a687d2-kube-api-access-5v6xs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.916584 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.916570    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.916738 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.916621    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/748c3069-9b31-49cd-b153-1897c2a687d2-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.916907 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.916885    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.916961 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.916929    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.917035 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.917016    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.918955 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.918935    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.919144 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.919127    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/748c3069-9b31-49cd-b153-1897c2a687d2-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:56.925323 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:56.925297    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v6xs\" (UniqueName: \"kubernetes.io/projected/748c3069-9b31-49cd-b153-1897c2a687d2-kube-api-access-5v6xs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") " pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:57.058097 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:57.058023    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:45:57.192717 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:57.192651    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"]
Jul 15 11:45:57.195515 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:45:57.195491    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748c3069_9b31_49cd_b153_1897c2a687d2.slice/crio-ee14ef59877eeb87931e99cd5cd863dcda2059439f7e327984820b08c8597104 WatchSource:0}: Error finding container ee14ef59877eeb87931e99cd5cd863dcda2059439f7e327984820b08c8597104: Status 404 returned error can't find the container with id ee14ef59877eeb87931e99cd5cd863dcda2059439f7e327984820b08c8597104
Jul 15 11:45:57.704132 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:57.704103    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l" event={"ID":"748c3069-9b31-49cd-b153-1897c2a687d2","Type":"ContainerStarted","Data":"3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21"}
Jul 15 11:45:57.704309 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:57.704150    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l" event={"ID":"748c3069-9b31-49cd-b153-1897c2a687d2","Type":"ContainerStarted","Data":"ee14ef59877eeb87931e99cd5cd863dcda2059439f7e327984820b08c8597104"}
Jul 15 11:45:57.723050 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:57.723004    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l" podStartSLOduration=1.722974508 podStartE2EDuration="1.722974508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:45:57.722047394 +0000 UTC m=+3635.521520920" watchObservedRunningTime="2026-07-15 11:45:57.722974508 +0000 UTC m=+3635.522448034"
Jul 15 11:45:58.058374 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:45:58.058343    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:46:07.058367 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:46:07.058332    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:46:07.070687 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:46:07.070660    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:46:07.760500 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:46:07.760474    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:54:34.399963 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:34.399886    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"]
Jul 15 11:54:49.420562 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.420503    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l" podUID="748c3069-9b31-49cd-b153-1897c2a687d2" containerName="main" containerID="cri-o://3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21" gracePeriod=45
Jul 15 11:54:49.673665 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.673613    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:54:49.793563 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.793538    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-tmp-dir\") pod \"748c3069-9b31-49cd-b153-1897c2a687d2\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") "
Jul 15 11:54:49.793563 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.793569    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/748c3069-9b31-49cd-b153-1897c2a687d2-tls-certs\") pod \"748c3069-9b31-49cd-b153-1897c2a687d2\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") "
Jul 15 11:54:49.793802 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.793726    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-model-cache\") pod \"748c3069-9b31-49cd-b153-1897c2a687d2\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") "
Jul 15 11:54:49.793802 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.793768    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-dshm\") pod \"748c3069-9b31-49cd-b153-1897c2a687d2\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") "
Jul 15 11:54:49.793910 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.793814    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v6xs\" (UniqueName: \"kubernetes.io/projected/748c3069-9b31-49cd-b153-1897c2a687d2-kube-api-access-5v6xs\") pod \"748c3069-9b31-49cd-b153-1897c2a687d2\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") "
Jul 15 11:54:49.793910 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.793816    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "748c3069-9b31-49cd-b153-1897c2a687d2" (UID: "748c3069-9b31-49cd-b153-1897c2a687d2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:54:49.793910 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.793839    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-home\") pod \"748c3069-9b31-49cd-b153-1897c2a687d2\" (UID: \"748c3069-9b31-49cd-b153-1897c2a687d2\") "
Jul 15 11:54:49.794054 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.793966    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-model-cache" (OuterVolumeSpecName: "model-cache") pod "748c3069-9b31-49cd-b153-1897c2a687d2" (UID: "748c3069-9b31-49cd-b153-1897c2a687d2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:54:49.794140 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.794125    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:54:49.794191 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.794146    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:54:49.794191 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.794156    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-home" (OuterVolumeSpecName: "home") pod "748c3069-9b31-49cd-b153-1897c2a687d2" (UID: "748c3069-9b31-49cd-b153-1897c2a687d2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:54:49.795836 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.795811    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/748c3069-9b31-49cd-b153-1897c2a687d2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "748c3069-9b31-49cd-b153-1897c2a687d2" (UID: "748c3069-9b31-49cd-b153-1897c2a687d2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:54:49.796018 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.795992    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/748c3069-9b31-49cd-b153-1897c2a687d2-kube-api-access-5v6xs" (OuterVolumeSpecName: "kube-api-access-5v6xs") pod "748c3069-9b31-49cd-b153-1897c2a687d2" (UID: "748c3069-9b31-49cd-b153-1897c2a687d2"). InnerVolumeSpecName "kube-api-access-5v6xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:54:49.796106 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.796092    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-dshm" (OuterVolumeSpecName: "dshm") pod "748c3069-9b31-49cd-b153-1897c2a687d2" (UID: "748c3069-9b31-49cd-b153-1897c2a687d2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:54:49.894677 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.894642    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:54:49.894677 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.894672    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v6xs\" (UniqueName: \"kubernetes.io/projected/748c3069-9b31-49cd-b153-1897c2a687d2-kube-api-access-5v6xs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:54:49.894677 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.894685    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/748c3069-9b31-49cd-b153-1897c2a687d2-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:54:49.894945 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:49.894698    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/748c3069-9b31-49cd-b153-1897c2a687d2-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:54:50.110383 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:50.110344    2536 generic.go:356] "Generic (PLEG): container finished" podID="748c3069-9b31-49cd-b153-1897c2a687d2" containerID="3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21" exitCode=0
Jul 15 11:54:50.110550 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:50.110418    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"
Jul 15 11:54:50.110550 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:50.110434    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l" event={"ID":"748c3069-9b31-49cd-b153-1897c2a687d2","Type":"ContainerDied","Data":"3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21"}
Jul 15 11:54:50.110550 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:50.110514    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l" event={"ID":"748c3069-9b31-49cd-b153-1897c2a687d2","Type":"ContainerDied","Data":"ee14ef59877eeb87931e99cd5cd863dcda2059439f7e327984820b08c8597104"}
Jul 15 11:54:50.110550 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:50.110536    2536 scope.go:117] "RemoveContainer" containerID="3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21"
Jul 15 11:54:50.119619 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:50.119602    2536 scope.go:117] "RemoveContainer" containerID="3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21"
Jul 15 11:54:50.119885 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:54:50.119860    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21\": container with ID starting with 3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21 not found: ID does not exist" containerID="3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21"
Jul 15 11:54:50.119976 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:50.119897    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21"} err="failed to get container status \"3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21\": rpc error: code = NotFound desc = could not find container \"3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21\": container with ID starting with 3ad8cdc8fabb497c2aa38884ed0351eae982fdd29991d81660e14fdd7dd99b21 not found: ID does not exist"
Jul 15 11:54:50.143788 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:50.143764    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"]
Jul 15 11:54:50.147709 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:50.147692    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-hcz7l"]
Jul 15 11:54:50.595557 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:54:50.595527    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="748c3069-9b31-49cd-b153-1897c2a687d2" path="/var/lib/kubelet/pods/748c3069-9b31-49cd-b153-1897c2a687d2/volumes"
Jul 15 11:55:03.980509 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:03.980472    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"]
Jul 15 11:55:03.981197 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:03.981170    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748c3069-9b31-49cd-b153-1897c2a687d2" containerName="main"
Jul 15 11:55:03.981385 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:03.981370    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="748c3069-9b31-49cd-b153-1897c2a687d2" containerName="main"
Jul 15 11:55:03.981695 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:03.981679    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="748c3069-9b31-49cd-b153-1897c2a687d2" containerName="main"
Jul 15 11:55:03.985888 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:03.985868    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:03.992964 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:03.992766    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Jul 15 11:55:03.992964 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:03.992855    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:55:03.992964 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:03.992882    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:55:03.993398 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:03.993378    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:55:03.995530 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:03.995506    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"]
Jul 15 11:55:04.113427 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.113401    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.113427 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.113433    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.113607 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.113497    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.113607 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.113533    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.113607 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.113575    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.113735 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.113611    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bd9n\" (UniqueName: \"kubernetes.io/projected/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-kube-api-access-9bd9n\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.214718 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.214693    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.214865 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.214736    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.214865 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.214773    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.214865 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.214829    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bd9n\" (UniqueName: \"kubernetes.io/projected/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-kube-api-access-9bd9n\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.215045 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.214865    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.215045 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.214890    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.215154 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.215125    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-home\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.215218 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.215168    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.215218 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.215204    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.217299 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.217241    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.217635 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.217614    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.230129 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.230106    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bd9n\" (UniqueName: \"kubernetes.io/projected/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-kube-api-access-9bd9n\") pod \"scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") " pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.297431 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.297361    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:04.437362 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:04.437334    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"]
Jul 15 11:55:04.439775 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:55:04.439746    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a9336b5_efc2_433f_85a9_f9d02ecb2a54.slice/crio-a38537769702e4d31c547b1e1ca731327019f2224b2ec8e3cc6e13523554e586 WatchSource:0}: Error finding container a38537769702e4d31c547b1e1ca731327019f2224b2ec8e3cc6e13523554e586: Status 404 returned error can't find the container with id a38537769702e4d31c547b1e1ca731327019f2224b2ec8e3cc6e13523554e586
Jul 15 11:55:05.183077 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:05.183039    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b" event={"ID":"0a9336b5-efc2-433f-85a9-f9d02ecb2a54","Type":"ContainerStarted","Data":"e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d"}
Jul 15 11:55:05.183511 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:05.183090    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b" event={"ID":"0a9336b5-efc2-433f-85a9-f9d02ecb2a54","Type":"ContainerStarted","Data":"a38537769702e4d31c547b1e1ca731327019f2224b2ec8e3cc6e13523554e586"}
Jul 15 11:55:05.210185 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:05.210138    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b" podStartSLOduration=2.210123995 podStartE2EDuration="2.210123995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:55:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:55:05.205744015 +0000 UTC m=+4183.005217521" watchObservedRunningTime="2026-07-15 11:55:05.210123995 +0000 UTC m=+4183.009597675"
Jul 15 11:55:05.298016 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:05.297976    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:14.297624 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:14.297587    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:14.313218 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:14.313190    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:55:15.242315 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:55:15.242268    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:56:38.709753 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:38.709717    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"]
Jul 15 11:56:53.729581 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:53.729513    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b" podUID="0a9336b5-efc2-433f-85a9-f9d02ecb2a54" containerName="main" containerID="cri-o://e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d" gracePeriod=45
Jul 15 11:56:53.979884 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:53.979829    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:56:54.078356 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.078325    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-home\") pod \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") "
Jul 15 11:56:54.078356 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.078361    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-model-cache\") pod \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") "
Jul 15 11:56:54.078578 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.078409    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-dshm\") pod \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") "
Jul 15 11:56:54.078578 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.078428    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tls-certs\") pod \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") "
Jul 15 11:56:54.078578 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.078456    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tmp-dir\") pod \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") "
Jul 15 11:56:54.078738 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.078631    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-home" (OuterVolumeSpecName: "home") pod "0a9336b5-efc2-433f-85a9-f9d02ecb2a54" (UID: "0a9336b5-efc2-433f-85a9-f9d02ecb2a54"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:56:54.078738 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.078659    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bd9n\" (UniqueName: \"kubernetes.io/projected/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-kube-api-access-9bd9n\") pod \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\" (UID: \"0a9336b5-efc2-433f-85a9-f9d02ecb2a54\") "
Jul 15 11:56:54.078850 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.078771    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-model-cache" (OuterVolumeSpecName: "model-cache") pod "0a9336b5-efc2-433f-85a9-f9d02ecb2a54" (UID: "0a9336b5-efc2-433f-85a9-f9d02ecb2a54"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:56:54.078850 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.078785    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0a9336b5-efc2-433f-85a9-f9d02ecb2a54" (UID: "0a9336b5-efc2-433f-85a9-f9d02ecb2a54"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:56:54.079091 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.079059    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:56:54.079228 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.079095    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:56:54.079228 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.079108    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:56:54.080679 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.080648    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-dshm" (OuterVolumeSpecName: "dshm") pod "0a9336b5-efc2-433f-85a9-f9d02ecb2a54" (UID: "0a9336b5-efc2-433f-85a9-f9d02ecb2a54"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 11:56:54.080778 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.080739    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0a9336b5-efc2-433f-85a9-f9d02ecb2a54" (UID: "0a9336b5-efc2-433f-85a9-f9d02ecb2a54"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 11:56:54.080934 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.080916    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-kube-api-access-9bd9n" (OuterVolumeSpecName: "kube-api-access-9bd9n") pod "0a9336b5-efc2-433f-85a9-f9d02ecb2a54" (UID: "0a9336b5-efc2-433f-85a9-f9d02ecb2a54"). InnerVolumeSpecName "kube-api-access-9bd9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 11:56:54.180579 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.180545    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bd9n\" (UniqueName: \"kubernetes.io/projected/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-kube-api-access-9bd9n\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:56:54.180579 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.180577    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:56:54.180779 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.180589    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a9336b5-efc2-433f-85a9-f9d02ecb2a54-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 11:56:54.679242 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.679202    2536 generic.go:356] "Generic (PLEG): container finished" podID="0a9336b5-efc2-433f-85a9-f9d02ecb2a54" containerID="e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d" exitCode=0
Jul 15 11:56:54.679242 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.679246    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"
Jul 15 11:56:54.679441 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.679298    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b" event={"ID":"0a9336b5-efc2-433f-85a9-f9d02ecb2a54","Type":"ContainerDied","Data":"e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d"}
Jul 15 11:56:54.679441 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.679352    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b" event={"ID":"0a9336b5-efc2-433f-85a9-f9d02ecb2a54","Type":"ContainerDied","Data":"a38537769702e4d31c547b1e1ca731327019f2224b2ec8e3cc6e13523554e586"}
Jul 15 11:56:54.679441 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.679383    2536 scope.go:117] "RemoveContainer" containerID="e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d"
Jul 15 11:56:54.687962 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.687945    2536 scope.go:117] "RemoveContainer" containerID="e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d"
Jul 15 11:56:54.688233 ip-10-0-134-178 kubenswrapper[2536]: E0715 11:56:54.688214    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d\": container with ID starting with e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d not found: ID does not exist" containerID="e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d"
Jul 15 11:56:54.688291 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.688241    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d"} err="failed to get container status \"e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d\": rpc error: code = NotFound desc = could not find container \"e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d\": container with ID starting with e8b9458320e1683d262883fbf86030cbde37259cb6281e6cdc2a6b2c4081b17d not found: ID does not exist"
Jul 15 11:56:54.700182 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.700151    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"]
Jul 15 11:56:54.706897 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:54.706875    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5pdw8b"]
Jul 15 11:56:56.594821 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:56:56.594786    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a9336b5-efc2-433f-85a9-f9d02ecb2a54" path="/var/lib/kubelet/pods/0a9336b5-efc2-433f-85a9-f9d02ecb2a54/volumes"
Jul 15 11:57:17.457847 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.457762    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"]
Jul 15 11:57:17.458507 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.458257    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a9336b5-efc2-433f-85a9-f9d02ecb2a54" containerName="main"
Jul 15 11:57:17.458507 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.458291    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a9336b5-efc2-433f-85a9-f9d02ecb2a54" containerName="main"
Jul 15 11:57:17.458507 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.458384    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a9336b5-efc2-433f-85a9-f9d02ecb2a54" containerName="main"
Jul 15 11:57:17.461360 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.461343    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:17.464592 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.464572    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Jul 15 11:57:17.464704 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.464620    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 15 11:57:17.464704 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.464579    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-p7tbn\""
Jul 15 11:57:17.464704 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.464656    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"rolling-upgrade-test-epp-sa-dockercfg-bh72x\""
Jul 15 11:57:17.469087 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.469071    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 15 11:57:17.477131 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.477108    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"]
Jul 15 11:57:17.580631 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.580603    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:17.580809 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.580687    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br7nn\" (UniqueName: \"kubernetes.io/projected/ae2f4b99-0667-404c-8d30-1202d0c09c2b-kube-api-access-br7nn\") pod \"rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:17.580809 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.580712    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tokenizer-uds\") pod \"rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:17.681819 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.681787    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br7nn\" (UniqueName: \"kubernetes.io/projected/ae2f4b99-0667-404c-8d30-1202d0c09c2b-kube-api-access-br7nn\") pod \"rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:17.681994 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.681828    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tokenizer-uds\") pod \"rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:17.681994 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.681855    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:17.682233 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.682216    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tokenizer-uds\") pod \"rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:17.684496 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.684466    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:17.692112 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.692084    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br7nn\" (UniqueName: \"kubernetes.io/projected/ae2f4b99-0667-404c-8d30-1202d0c09c2b-kube-api-access-br7nn\") pod \"rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:17.771386 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.771307    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:17.928019 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:17.927987    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"]
Jul 15 11:57:17.931990 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:57:17.931962    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2f4b99_0667_404c_8d30_1202d0c09c2b.slice/crio-d4d78d07679209b7a66fd10bb496d3040d97e54afd7b5efb0b64131a992adac7 WatchSource:0}: Error finding container d4d78d07679209b7a66fd10bb496d3040d97e54afd7b5efb0b64131a992adac7: Status 404 returned error can't find the container with id d4d78d07679209b7a66fd10bb496d3040d97e54afd7b5efb0b64131a992adac7
Jul 15 11:57:18.794927 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:18.794896    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz" event={"ID":"ae2f4b99-0667-404c-8d30-1202d0c09c2b","Type":"ContainerStarted","Data":"2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3"}
Jul 15 11:57:18.795357 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:18.794943    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz" event={"ID":"ae2f4b99-0667-404c-8d30-1202d0c09c2b","Type":"ContainerStarted","Data":"d4d78d07679209b7a66fd10bb496d3040d97e54afd7b5efb0b64131a992adac7"}
Jul 15 11:57:18.795357 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:18.795040    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:18.817106 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:18.817060    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz" podStartSLOduration=1.8170437339999999 podStartE2EDuration="1.817043734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:57:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:57:18.81459513 +0000 UTC m=+4316.614068666" watchObservedRunningTime="2026-07-15 11:57:18.817043734 +0000 UTC m=+4316.616517259"
Jul 15 11:57:49.801101 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:49.801064    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 11:57:51.924265 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:51.924224    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"]
Jul 15 11:57:51.928246 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:51.928227    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:51.940495 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:51.940471    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"]
Jul 15 11:57:51.979988 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:51.979959    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-home\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:51.980110 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:51.980009    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-model-cache\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:51.980110 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:51.980037    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tls-certs\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:51.980110 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:51.980066    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-dshm\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:51.980110 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:51.980090    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shr5r\" (UniqueName: \"kubernetes.io/projected/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-kube-api-access-shr5r\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:51.980252 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:51.980146    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tmp-dir\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.080861 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.080837    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shr5r\" (UniqueName: \"kubernetes.io/projected/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-kube-api-access-shr5r\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.080974 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.080882    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tmp-dir\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.080974 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.080929    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-home\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.081092 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.080979    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-model-cache\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.081092 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.080995    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tls-certs\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.081092 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.081019    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-dshm\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.081327 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.081307    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tmp-dir\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.081405 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.081369    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-home\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.081468 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.081428    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-model-cache\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.083516 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.083495    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-dshm\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.083785 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.083768    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tls-certs\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.089794 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.089772    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shr5r\" (UniqueName: \"kubernetes.io/projected/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-kube-api-access-shr5r\") pod \"rolling-upgrade-test-kserve-859df57bcb-wl8g6\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") " pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.238955 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.238927    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:57:52.379630 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.379595    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"]
Jul 15 11:57:52.382270 ip-10-0-134-178 kubenswrapper[2536]: W0715 11:57:52.382242    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ce1d2fb_4961_4791_b49f_d5c4c75d36f0.slice/crio-05425da55a7ec61c259fbf1bdc28009bb4ee429c8568f1c2952fd8fda06ebf50 WatchSource:0}: Error finding container 05425da55a7ec61c259fbf1bdc28009bb4ee429c8568f1c2952fd8fda06ebf50: Status 404 returned error can't find the container with id 05425da55a7ec61c259fbf1bdc28009bb4ee429c8568f1c2952fd8fda06ebf50
Jul 15 11:57:52.958036 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.958000    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6" event={"ID":"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0","Type":"ContainerStarted","Data":"ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21"}
Jul 15 11:57:52.958431 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.958051    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6" event={"ID":"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0","Type":"ContainerStarted","Data":"05425da55a7ec61c259fbf1bdc28009bb4ee429c8568f1c2952fd8fda06ebf50"}
Jul 15 11:57:52.978553 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:52.978504    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6" podStartSLOduration=1.978488707 podStartE2EDuration="1.978488707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 11:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 11:57:52.976411933 +0000 UTC m=+4350.775885467" watchObservedRunningTime="2026-07-15 11:57:52.978488707 +0000 UTC m=+4350.777962254"
Jul 15 11:57:53.239794 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:57:53.239720    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:58:02.239979 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:58:02.239941    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:58:02.252305 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:58:02.252263    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 11:58:03.014528 ip-10-0-134-178 kubenswrapper[2536]: I0715 11:58:03.014494    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 12:01:06.140806 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:06.140772    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"]
Jul 15 12:01:06.148204 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:06.148167    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"]
Jul 15 12:01:09.801209 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:09.801168    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz" podUID="ae2f4b99-0667-404c-8d30-1202d0c09c2b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 12:01:19.801369 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:19.801331    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz" podUID="ae2f4b99-0667-404c-8d30-1202d0c09c2b" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Jul 15 12:01:21.163494 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.163458    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6" podUID="5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" containerName="main" containerID="cri-o://ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21" gracePeriod=45
Jul 15 12:01:21.168657 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.168221    2536 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz" podUID="ae2f4b99-0667-404c-8d30-1202d0c09c2b" containerName="main" containerID="cri-o://2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3" gracePeriod=45
Jul 15 12:01:21.411974 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.411952    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 12:01:21.443597 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.443567    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 12:01:21.566890 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.566851    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tls-certs\") pod \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") "
Jul 15 12:01:21.566890 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.566911    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-br7nn\" (UniqueName: \"kubernetes.io/projected/ae2f4b99-0667-404c-8d30-1202d0c09c2b-kube-api-access-br7nn\") pod \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") "
Jul 15 12:01:21.567109 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.566948    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-dshm\") pod \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") "
Jul 15 12:01:21.567109 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.566990    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shr5r\" (UniqueName: \"kubernetes.io/projected/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-kube-api-access-shr5r\") pod \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") "
Jul 15 12:01:21.567109 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.567011    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tmp-dir\") pod \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") "
Jul 15 12:01:21.567109 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.567042    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tokenizer-uds\") pod \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\" (UID: \"ae2f4b99-0667-404c-8d30-1202d0c09c2b\") "
Jul 15 12:01:21.567109 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.567067    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-model-cache\") pod \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") "
Jul 15 12:01:21.567109 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.567092    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tls-certs\") pod \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") "
Jul 15 12:01:21.567442 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.567120    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-home\") pod \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\" (UID: \"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0\") "
Jul 15 12:01:21.567442 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.567331    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" (UID: "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:01:21.567555 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.567466    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-model-cache" (OuterVolumeSpecName: "model-cache") pod "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" (UID: "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:01:21.567610 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.567543    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-home" (OuterVolumeSpecName: "home") pod "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" (UID: "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:01:21.567610 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.567558    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "ae2f4b99-0667-404c-8d30-1202d0c09c2b" (UID: "ae2f4b99-0667-404c-8d30-1202d0c09c2b"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:01:21.567714 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.567487    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tmp-dir\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 12:01:21.569370 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.569346    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-dshm" (OuterVolumeSpecName: "dshm") pod "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" (UID: "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:01:21.569370 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.569365    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae2f4b99-0667-404c-8d30-1202d0c09c2b-kube-api-access-br7nn" (OuterVolumeSpecName: "kube-api-access-br7nn") pod "ae2f4b99-0667-404c-8d30-1202d0c09c2b" (UID: "ae2f4b99-0667-404c-8d30-1202d0c09c2b"). InnerVolumeSpecName "kube-api-access-br7nn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:01:21.569646 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.569627    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ae2f4b99-0667-404c-8d30-1202d0c09c2b" (UID: "ae2f4b99-0667-404c-8d30-1202d0c09c2b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:01:21.569717 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.569637    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-kube-api-access-shr5r" (OuterVolumeSpecName: "kube-api-access-shr5r") pod "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" (UID: "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0"). InnerVolumeSpecName "kube-api-access-shr5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:01:21.569757 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.569711    2536 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" (UID: "5ce1d2fb-4961-4791-b49f-d5c4c75d36f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:01:21.668302 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.668253    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shr5r\" (UniqueName: \"kubernetes.io/projected/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-kube-api-access-shr5r\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 12:01:21.668302 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.668300    2536 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tokenizer-uds\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 12:01:21.668302 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.668311    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-model-cache\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 12:01:21.668476 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.668319    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 12:01:21.668476 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.668326    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-home\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 12:01:21.668476 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.668333    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae2f4b99-0667-404c-8d30-1202d0c09c2b-tls-certs\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 12:01:21.668476 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.668340    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-br7nn\" (UniqueName: \"kubernetes.io/projected/ae2f4b99-0667-404c-8d30-1202d0c09c2b-kube-api-access-br7nn\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 12:01:21.668476 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.668348    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0-dshm\") on node \"ip-10-0-134-178.ec2.internal\" DevicePath \"\""
Jul 15 12:01:21.926703 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.926671    2536 generic.go:356] "Generic (PLEG): container finished" podID="ae2f4b99-0667-404c-8d30-1202d0c09c2b" containerID="2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3" exitCode=0
Jul 15 12:01:21.926869 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.926721    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"
Jul 15 12:01:21.926869 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.926765    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz" event={"ID":"ae2f4b99-0667-404c-8d30-1202d0c09c2b","Type":"ContainerDied","Data":"2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3"}
Jul 15 12:01:21.926869 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.926820    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz" event={"ID":"ae2f4b99-0667-404c-8d30-1202d0c09c2b","Type":"ContainerDied","Data":"d4d78d07679209b7a66fd10bb496d3040d97e54afd7b5efb0b64131a992adac7"}
Jul 15 12:01:21.926869 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.926847    2536 scope.go:117] "RemoveContainer" containerID="2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3"
Jul 15 12:01:21.928132 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.928106    2536 generic.go:356] "Generic (PLEG): container finished" podID="5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" containerID="ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21" exitCode=0
Jul 15 12:01:21.928252 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.928138    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6" event={"ID":"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0","Type":"ContainerDied","Data":"ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21"}
Jul 15 12:01:21.928252 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.928162    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"
Jul 15 12:01:21.928252 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.928189    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6" event={"ID":"5ce1d2fb-4961-4791-b49f-d5c4c75d36f0","Type":"ContainerDied","Data":"05425da55a7ec61c259fbf1bdc28009bb4ee429c8568f1c2952fd8fda06ebf50"}
Jul 15 12:01:21.936870 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.936852    2536 scope.go:117] "RemoveContainer" containerID="2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3"
Jul 15 12:01:21.937126 ip-10-0-134-178 kubenswrapper[2536]: E0715 12:01:21.937102    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3\": container with ID starting with 2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3 not found: ID does not exist" containerID="2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3"
Jul 15 12:01:21.937210 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.937138    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3"} err="failed to get container status \"2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3\": rpc error: code = NotFound desc = could not find container \"2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3\": container with ID starting with 2dd96b94706d7e8b13baaf61009350eb107d52bb9e260474481a5d6cad0ed1c3 not found: ID does not exist"
Jul 15 12:01:21.937210 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.937161    2536 scope.go:117] "RemoveContainer" containerID="ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21"
Jul 15 12:01:21.945375 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.945357    2536 scope.go:117] "RemoveContainer" containerID="ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21"
Jul 15 12:01:21.945610 ip-10-0-134-178 kubenswrapper[2536]: E0715 12:01:21.945590    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21\": container with ID starting with ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21 not found: ID does not exist" containerID="ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21"
Jul 15 12:01:21.945687 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.945620    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21"} err="failed to get container status \"ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21\": rpc error: code = NotFound desc = could not find container \"ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21\": container with ID starting with ce508a8cf0037a4b481ffa0daf7ceb795a607084f9b797f622ab0611710f5a21 not found: ID does not exist"
Jul 15 12:01:21.948541 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.948519    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"]
Jul 15 12:01:21.952835 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.952814    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-745dd558f-tqflz"]
Jul 15 12:01:21.966040 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.966018    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"]
Jul 15 12:01:21.973397 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:21.973379    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-wl8g6"]
Jul 15 12:01:22.594892 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:22.594857    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" path="/var/lib/kubelet/pods/5ce1d2fb-4961-4791-b49f-d5c4c75d36f0/volumes"
Jul 15 12:01:22.595235 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:01:22.595215    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae2f4b99-0667-404c-8d30-1202d0c09c2b" path="/var/lib/kubelet/pods/ae2f4b99-0667-404c-8d30-1202d0c09c2b/volumes"
Jul 15 12:06:55.095326 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:55.095299    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-jng46_55a8418e-50db-4c57-b760-be2777bedc3a/kuadrant-console-plugin/0.log"
Jul 15 12:06:55.373348 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:55.373241    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-frhbf_c0f17d07-73e9-4bab-b471-977e8074167f/manager/0.log"
Jul 15 12:06:55.383002 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:55.382968    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-p5c6m_664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee/limitador/0.log"
Jul 15 12:06:58.144795 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.144765    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lt44n/must-gather-d27px"]
Jul 15 12:06:58.145170 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.145073    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" containerName="main"
Jul 15 12:06:58.145170 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.145084    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" containerName="main"
Jul 15 12:06:58.145170 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.145099    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae2f4b99-0667-404c-8d30-1202d0c09c2b" containerName="main"
Jul 15 12:06:58.145170 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.145105    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae2f4b99-0667-404c-8d30-1202d0c09c2b" containerName="main"
Jul 15 12:06:58.145170 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.145164    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae2f4b99-0667-404c-8d30-1202d0c09c2b" containerName="main"
Jul 15 12:06:58.145170 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.145175    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ce1d2fb-4961-4791-b49f-d5c4c75d36f0" containerName="main"
Jul 15 12:06:58.148331 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.148309    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lt44n/must-gather-d27px"
Jul 15 12:06:58.150516 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.150491    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lt44n\"/\"default-dockercfg-rg9nd\""
Jul 15 12:06:58.150799 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.150777    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lt44n\"/\"openshift-service-ca.crt\""
Jul 15 12:06:58.150868 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.150849    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lt44n\"/\"kube-root-ca.crt\""
Jul 15 12:06:58.157312 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.157268    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lt44n/must-gather-d27px"]
Jul 15 12:06:58.205909 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.205881    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t25dp\" (UniqueName: \"kubernetes.io/projected/af439953-a83c-4338-8caf-ec668cde8af4-kube-api-access-t25dp\") pod \"must-gather-d27px\" (UID: \"af439953-a83c-4338-8caf-ec668cde8af4\") " pod="openshift-must-gather-lt44n/must-gather-d27px"
Jul 15 12:06:58.206020 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.205927    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/af439953-a83c-4338-8caf-ec668cde8af4-must-gather-output\") pod \"must-gather-d27px\" (UID: \"af439953-a83c-4338-8caf-ec668cde8af4\") " pod="openshift-must-gather-lt44n/must-gather-d27px"
Jul 15 12:06:58.307345 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.307314    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/af439953-a83c-4338-8caf-ec668cde8af4-must-gather-output\") pod \"must-gather-d27px\" (UID: \"af439953-a83c-4338-8caf-ec668cde8af4\") " pod="openshift-must-gather-lt44n/must-gather-d27px"
Jul 15 12:06:58.307473 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.307391    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t25dp\" (UniqueName: \"kubernetes.io/projected/af439953-a83c-4338-8caf-ec668cde8af4-kube-api-access-t25dp\") pod \"must-gather-d27px\" (UID: \"af439953-a83c-4338-8caf-ec668cde8af4\") " pod="openshift-must-gather-lt44n/must-gather-d27px"
Jul 15 12:06:58.307651 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.307632    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/af439953-a83c-4338-8caf-ec668cde8af4-must-gather-output\") pod \"must-gather-d27px\" (UID: \"af439953-a83c-4338-8caf-ec668cde8af4\") " pod="openshift-must-gather-lt44n/must-gather-d27px"
Jul 15 12:06:58.335676 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.335647    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t25dp\" (UniqueName: \"kubernetes.io/projected/af439953-a83c-4338-8caf-ec668cde8af4-kube-api-access-t25dp\") pod \"must-gather-d27px\" (UID: \"af439953-a83c-4338-8caf-ec668cde8af4\") " pod="openshift-must-gather-lt44n/must-gather-d27px"
Jul 15 12:06:58.458877 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.458857    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lt44n/must-gather-d27px"
Jul 15 12:06:58.593293 ip-10-0-134-178 kubenswrapper[2536]: W0715 12:06:58.593245    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf439953_a83c_4338_8caf_ec668cde8af4.slice/crio-6047d9821a2bdda4f9192a54da41142a21c48bc819a7988c13f0c0a4c2f46987 WatchSource:0}: Error finding container 6047d9821a2bdda4f9192a54da41142a21c48bc819a7988c13f0c0a4c2f46987: Status 404 returned error can't find the container with id 6047d9821a2bdda4f9192a54da41142a21c48bc819a7988c13f0c0a4c2f46987
Jul 15 12:06:58.595447 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.595426    2536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:06:58.596829 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:58.596805    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lt44n/must-gather-d27px"]
Jul 15 12:06:59.403949 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:06:59.403913    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lt44n/must-gather-d27px" event={"ID":"af439953-a83c-4338-8caf-ec668cde8af4","Type":"ContainerStarted","Data":"6047d9821a2bdda4f9192a54da41142a21c48bc819a7988c13f0c0a4c2f46987"}
Jul 15 12:07:00.413127 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:00.413051    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lt44n/must-gather-d27px" event={"ID":"af439953-a83c-4338-8caf-ec668cde8af4","Type":"ContainerStarted","Data":"3dfd8c60599611fa81bf8e0f98fdf4d60c44f4b4862085724ace25df6e9b7cb0"}
Jul 15 12:07:00.413127 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:00.413100    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lt44n/must-gather-d27px" event={"ID":"af439953-a83c-4338-8caf-ec668cde8af4","Type":"ContainerStarted","Data":"4876728fb0f48b0d550f96cca105fbef3c3f3f542058eb4b9a31131195e4e8a1"}
Jul 15 12:07:00.433302 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:00.433229    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lt44n/must-gather-d27px" podStartSLOduration=1.499884676 podStartE2EDuration="2.43321211s" totalImagesPullingTime="933.327434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:06:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:07:00.432434795 +0000 UTC m=+4898.231908321" watchObservedRunningTime="2026-07-15 12:07:00.43321211 +0000 UTC m=+4898.232685667"
Jul 15 12:07:01.108431 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:01.108400    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cq9p2_f40144cc-a5a5-47a0-8e2a-8de3ddde2ad5/global-pull-secret-syncer/0.log"
Jul 15 12:07:01.230257 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:01.230217    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rbbsd_c191b9b2-e426-494e-8dbb-66d9da189ec3/konnectivity-agent/0.log"
Jul 15 12:07:01.374397 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:01.374324    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-178.ec2.internal_da1d1b399f6297ad7ed288ac69e2c3f1/haproxy/0.log"
Jul 15 12:07:05.868257 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:05.868179    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-85fd4b8b9d-jng46_55a8418e-50db-4c57-b760-be2777bedc3a/kuadrant-console-plugin/0.log"
Jul 15 12:07:06.244384 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:06.244353    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-bb7ddb7d7-frhbf_c0f17d07-73e9-4bab-b471-977e8074167f/manager/0.log"
Jul 15 12:07:06.271254 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:06.271222    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-p5c6m_664e3f9c-a7b3-4eeb-8c35-a72a9a5410ee/limitador/0.log"
Jul 15 12:07:07.611585 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:07.611496    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80eb6dc6-c3c9-45e1-b979-5422a1f92c72/alertmanager/0.log"
Jul 15 12:07:07.642810 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:07.642698    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80eb6dc6-c3c9-45e1-b979-5422a1f92c72/config-reloader/0.log"
Jul 15 12:07:07.667321 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:07.667260    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80eb6dc6-c3c9-45e1-b979-5422a1f92c72/kube-rbac-proxy-web/0.log"
Jul 15 12:07:07.699657 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:07.699551    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80eb6dc6-c3c9-45e1-b979-5422a1f92c72/kube-rbac-proxy/0.log"
Jul 15 12:07:07.728533 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:07.728462    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80eb6dc6-c3c9-45e1-b979-5422a1f92c72/kube-rbac-proxy-metric/0.log"
Jul 15 12:07:07.751478 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:07.751366    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80eb6dc6-c3c9-45e1-b979-5422a1f92c72/prom-label-proxy/0.log"
Jul 15 12:07:07.775471 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:07.775370    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80eb6dc6-c3c9-45e1-b979-5422a1f92c72/init-config-reloader/0.log"
Jul 15 12:07:07.835471 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:07.835440    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-xn2p6_c9493868-2500-412f-bfcc-f02972b330a3/cluster-monitoring-operator/0.log"
Jul 15 12:07:08.311019 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.310984    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zv49m_363ab220-4030-4496-b9ed-9e86a7de501c/node-exporter/0.log"
Jul 15 12:07:08.341589 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.341558    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zv49m_363ab220-4030-4496-b9ed-9e86a7de501c/kube-rbac-proxy/0.log"
Jul 15 12:07:08.367555 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.367508    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zv49m_363ab220-4030-4496-b9ed-9e86a7de501c/init-textfile/0.log"
Jul 15 12:07:08.399975 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.399921    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-m5npz_8f89c65b-cada-4f06-b034-72deee0905f8/kube-rbac-proxy-main/0.log"
Jul 15 12:07:08.425382 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.425324    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-m5npz_8f89c65b-cada-4f06-b034-72deee0905f8/kube-rbac-proxy-self/0.log"
Jul 15 12:07:08.449233 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.449202    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-m5npz_8f89c65b-cada-4f06-b034-72deee0905f8/openshift-state-metrics/0.log"
Jul 15 12:07:08.497023 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.496993    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09/prometheus/0.log"
Jul 15 12:07:08.533168 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.533135    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09/config-reloader/0.log"
Jul 15 12:07:08.575721 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.575671    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09/thanos-sidecar/0.log"
Jul 15 12:07:08.608313 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.608157    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09/kube-rbac-proxy-web/0.log"
Jul 15 12:07:08.643403 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.643374    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09/kube-rbac-proxy/0.log"
Jul 15 12:07:08.668598 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.668573    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09/kube-rbac-proxy-thanos/0.log"
Jul 15 12:07:08.696525 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.696464    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bddbe3a4-0f70-4a0e-ad4d-d21b1e40de09/init-config-reloader/0.log"
Jul 15 12:07:08.910796 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.910715    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64555f4986-xjv87_fe873f8f-40ba-42db-9b07-99cf1d0562f4/telemeter-client/0.log"
Jul 15 12:07:08.991166 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:08.991139    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64555f4986-xjv87_fe873f8f-40ba-42db-9b07-99cf1d0562f4/reload/0.log"
Jul 15 12:07:09.056058 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:09.056013    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-64555f4986-xjv87_fe873f8f-40ba-42db-9b07-99cf1d0562f4/kube-rbac-proxy/0.log"
Jul 15 12:07:10.421875 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:10.421831    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-d68nc_470d651d-0c05-4629-a564-50a940da19d1/networking-console-plugin/0.log"
Jul 15 12:07:10.922915 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:10.922877    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"]
Jul 15 12:07:10.928204 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:10.928179    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:10.937641 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:10.937580    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"]
Jul 15 12:07:11.038716 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.038683    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-proc\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.038896 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.038784    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-podres\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.038896 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.038852    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-sys\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.038896 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.038875    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-lib-modules\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.039050 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.038904    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krft5\" (UniqueName: \"kubernetes.io/projected/5c2814de-e82e-4657-bbc4-4b8962b237d4-kube-api-access-krft5\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.140099 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.140066    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-proc\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.140312 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.140109    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-podres\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.140312 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.140149    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-sys\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.140312 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.140172    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-lib-modules\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.140312 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.140201    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krft5\" (UniqueName: \"kubernetes.io/projected/5c2814de-e82e-4657-bbc4-4b8962b237d4-kube-api-access-krft5\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.140312 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.140288    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-sys\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.140603 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.140290    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-podres\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.140603 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.140362    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-proc\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.140603 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.140378    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c2814de-e82e-4657-bbc4-4b8962b237d4-lib-modules\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.149961 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.149927    2536 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krft5\" (UniqueName: \"kubernetes.io/projected/5c2814de-e82e-4657-bbc4-4b8962b237d4-kube-api-access-krft5\") pod \"perf-node-gather-daemonset-w5lxq\" (UID: \"5c2814de-e82e-4657-bbc4-4b8962b237d4\") " pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.240026 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.239998    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:11.421772 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.421746    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"]
Jul 15 12:07:11.424678 ip-10-0-134-178 kubenswrapper[2536]: W0715 12:07:11.424653    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5c2814de_e82e_4657_bbc4_4b8962b237d4.slice/crio-7286180b8baece93c21bd04dc51662763c597df07f73248063fad52c2ec9b7c8 WatchSource:0}: Error finding container 7286180b8baece93c21bd04dc51662763c597df07f73248063fad52c2ec9b7c8: Status 404 returned error can't find the container with id 7286180b8baece93c21bd04dc51662763c597df07f73248063fad52c2ec9b7c8
Jul 15 12:07:11.434073 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.433685    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5bbbfb8bbc-d2cf8_6220d036-3421-4865-8c0e-33a51551ec79/console/0.log"
Jul 15 12:07:11.477606 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.477566    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-znd4t_bf9442c2-785b-449d-8cea-dee4b0564a08/download-server/0.log"
Jul 15 12:07:11.484443 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:11.484415    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq" event={"ID":"5c2814de-e82e-4657-bbc4-4b8962b237d4","Type":"ContainerStarted","Data":"7286180b8baece93c21bd04dc51662763c597df07f73248063fad52c2ec9b7c8"}
Jul 15 12:07:12.493239 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:12.493197    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq" event={"ID":"5c2814de-e82e-4657-bbc4-4b8962b237d4","Type":"ContainerStarted","Data":"f74d2e964f0b53dc5655acb86203b9142973bc61253eb83f86fc04da24d21ed9"}
Jul 15 12:07:12.493737 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:12.493343    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:12.513109 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:12.513052    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq" podStartSLOduration=2.513037251 podStartE2EDuration="2.513037251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:07:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:07:12.510362148 +0000 UTC m=+4910.309835675" watchObservedRunningTime="2026-07-15 12:07:12.513037251 +0000 UTC m=+4910.312510776"
Jul 15 12:07:13.072978 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:13.072943    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-52trx_3b2f4d3f-9e4f-470d-88ab-59c274848f26/dns/0.log"
Jul 15 12:07:13.094456 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:13.094432    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-52trx_3b2f4d3f-9e4f-470d-88ab-59c274848f26/kube-rbac-proxy/0.log"
Jul 15 12:07:13.261681 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:13.261648    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s986k_764e82f9-85c4-44bb-9a80-72c5e55a953b/dns-node-resolver/0.log"
Jul 15 12:07:13.970858 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:13.970830    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gx9mp_9ad1c6a7-84e8-423b-a1bf-bbb2200a0faf/node-ca/0.log"
Jul 15 12:07:16.366711 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:16.366687    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kbdbx_1c608745-d6fc-4ed0-9916-9f777692890c/serve-healthcheck-canary/0.log"
Jul 15 12:07:16.879107 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:16.879077    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-tcwtw_c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9/insights-operator/0.log"
Jul 15 12:07:16.879944 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:16.879922    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-tcwtw_c7c27c79-058c-4b3b-a5f4-c72f0bf2b2d9/insights-operator/1.log"
Jul 15 12:07:16.976790 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:16.976759    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vlj6f_b144126b-086e-4cea-bc1a-3df5c25b8c35/kube-rbac-proxy/0.log"
Jul 15 12:07:16.997877 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:16.997849    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vlj6f_b144126b-086e-4cea-bc1a-3df5c25b8c35/exporter/0.log"
Jul 15 12:07:17.029540 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:17.029516    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vlj6f_b144126b-086e-4cea-bc1a-3df5c25b8c35/extractor/0.log"
Jul 15 12:07:18.509431 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:18.509400    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lt44n/perf-node-gather-daemonset-w5lxq"
Jul 15 12:07:20.019532 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:20.019503    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d9b94dcd4-4tnld_9119c75d-106d-4de2-b099-2f1088e0a199/manager/0.log"
Jul 15 12:07:20.077173 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:20.077145    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-5mzxr_3f73f68b-76a5-4ece-9bf1-1af8efaf49e0/openshift-lws-operator/0.log"
Jul 15 12:07:28.585075 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:28.585039    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjftd_7950898a-8257-4b97-9fed-66ceafd63b56/kube-multus-additional-cni-plugins/0.log"
Jul 15 12:07:28.610787 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:28.610751    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjftd_7950898a-8257-4b97-9fed-66ceafd63b56/egress-router-binary-copy/0.log"
Jul 15 12:07:28.639571 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:28.639538    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjftd_7950898a-8257-4b97-9fed-66ceafd63b56/cni-plugins/0.log"
Jul 15 12:07:28.667784 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:28.667756    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjftd_7950898a-8257-4b97-9fed-66ceafd63b56/bond-cni-plugin/0.log"
Jul 15 12:07:28.691269 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:28.691246    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjftd_7950898a-8257-4b97-9fed-66ceafd63b56/routeoverride-cni/0.log"
Jul 15 12:07:28.714228 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:28.714203    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjftd_7950898a-8257-4b97-9fed-66ceafd63b56/whereabouts-cni-bincopy/0.log"
Jul 15 12:07:28.738906 ip-10-0-134-178 kubenswrapper[2536]: I0715 12:07:28.738881    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjftd_7950898a-8257-4b97-9fed-66ceafd63b56/whereabouts-cni/0.log"