Jul 14 15:46:56.360767 ip-10-0-135-20 systemd[1]: Starting Kubernetes Kubelet...
Jul 14 15:46:56.770543 ip-10-0-135-20 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 15:46:56.770543 ip-10-0-135-20 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 14 15:46:56.770543 ip-10-0-135-20 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 15:46:56.770543 ip-10-0-135-20 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 14 15:46:56.770543 ip-10-0-135-20 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 15:46:56.773285 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.773197    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 14 15:46:56.781522 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781490    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 15:46:56.781522 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781511    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 15:46:56.781522 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781515    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 15:46:56.781522 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781518    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 15:46:56.781522 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781521    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 15:46:56.781522 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781525    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 15:46:56.781522 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781528    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 15:46:56.781522 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781531    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 15:46:56.781522 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781534    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781537    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781539    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781542    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781545    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781547    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781549    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781552    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781554    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781557    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781559    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781562    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781564    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781566    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781569    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781571    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781574    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781576    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781579    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781581    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 15:46:56.781823 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781584    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781586    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781589    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781598    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781601    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781603    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781606    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781609    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781611    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781614    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781617    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781619    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781622    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781624    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781627    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781630    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781632    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781635    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781637    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781640    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 15:46:56.782323 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781643    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781645    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781648    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781650    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781653    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781655    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781658    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781660    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781662    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781665    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781667    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781670    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781673    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781676    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781680    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781684    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781687    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781691    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 15:46:56.782846 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781695    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781697    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781700    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781703    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781706    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781709    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781711    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781714    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781718    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781721    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781725    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781728    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781731    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781734    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781737    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781739    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781742    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781745    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781748    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781750    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 15:46:56.783268 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781753    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781755    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781758    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781760    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781764    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781766    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781768    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781771    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781773    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781776    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781778    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781781    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.781783    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782511    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782519    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782522    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782525    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782527    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782530    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 15:46:56.783865 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782533    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782536    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782539    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782542    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782544    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782547    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782550    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782552    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782555    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782558    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782561    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782564    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782566    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782569    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782572    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782575    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782577    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782580    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782582    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782585    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 15:46:56.784347 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782587    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782590    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782595    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782598    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782601    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782604    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782607    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782610    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782612    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782614    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782617    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782620    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782624    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782628    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782631    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782634    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782636    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782639    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782641    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 15:46:56.784886 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782644    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782646    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782649    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782651    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782654    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782657    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782659    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782661    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782664    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782666    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782668    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782671    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782673    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782675    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782677    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782680    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782683    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782685    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782689    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 15:46:56.785345 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782692    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782695    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782698    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782700    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782702    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782705    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782707    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782711    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782714    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782716    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782719    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782721    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782724    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782727    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782730    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782733    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782735    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782737    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782740    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782743    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 15:46:56.785822 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782745    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782747    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782751    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782754    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782756    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782758    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782761    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782763    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782766    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782768    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782771    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782773    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782776    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782778    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.782781    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783156    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783160    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783163    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783166    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783169    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 15:46:56.786320 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783172    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783175    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783177    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783180    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783183    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783186    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783188    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783191    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783193    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783196    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783198    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783201    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783203    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783208    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783211    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783214    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783217    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783220    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783223    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 15:46:56.786819 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783226    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783228    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783231    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783234    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783237    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783239    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783242    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783244    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783247    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783250    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783252    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783255    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783258    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783260    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783263    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783265    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783268    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783270    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783273    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783278    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 15:46:56.787275 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783281    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783284    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783286    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783289    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783292    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783295    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783297    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783300    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783302    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783305    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783307    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783309    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783312    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783314    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783316    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783319    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783322    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783324    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783326    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783329    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 15:46:56.787775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783331    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783334    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783336    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783340    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783343    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783346    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783348    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783351    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783353    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783355    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783359    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783362    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783365    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783367    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783370    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783373    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783375    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783378    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783381    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 15:46:56.788256 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783383    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783385    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783388    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783390    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783393    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783395    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783397    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783400    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783402    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783404    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783407    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783409    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783412    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783414    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783416    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.783418    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783522    2560 flags.go:64] FLAG: --address="0.0.0.0"
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783530    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783536    2560 flags.go:64] FLAG: --anonymous-auth="true"
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783541    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783545    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 14 15:46:56.788723 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783548    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783552    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783556    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783559    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783562    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783565    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783568    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783572    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783575    2560 flags.go:64] FLAG: --cgroup-root=""
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783577    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783580    2560 flags.go:64] FLAG: --client-ca-file=""
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783584    2560 flags.go:64] FLAG: --cloud-provider="external"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783587    2560 flags.go:64] FLAG: --cluster-dns="[]"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783591    2560 flags.go:64] FLAG: --cluster-domain=""
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783594    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783597    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783601    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783606    2560 flags.go:64] FLAG: --container-log-max-files="5"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783612    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783617    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783621    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783625    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783628    2560 flags.go:64] FLAG: --contention-profiling="false"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783631    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 14 15:46:56.789211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783634    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783637    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783640    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783644    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783650    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783653    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783656    2560 flags.go:64] FLAG: --enable-load-reader="false"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783659    2560 flags.go:64] FLAG: --enable-server="true"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783662    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783666    2560 flags.go:64] FLAG: --event-burst="100"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783669    2560 flags.go:64] FLAG: --event-qps="50"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783672    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783674    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783677    2560 flags.go:64] FLAG: --eviction-hard=""
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783681    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783684    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783687    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783689    2560 flags.go:64] FLAG: --eviction-soft=""
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783692    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783695    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783702    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783705    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783708    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783710    2560 flags.go:64] FLAG: --fail-swap-on="true"
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783713    2560 flags.go:64] FLAG: --feature-gates=""
Jul 14 15:46:56.789799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783717    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783721    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783724    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783727    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783729    2560 flags.go:64] FLAG: --healthz-port="10248"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783732    2560 flags.go:64] FLAG: --help="false"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783735    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-135-20.ec2.internal"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783738    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783741    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783744    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783747    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783752    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783755    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783758    2560 flags.go:64] FLAG: --image-service-endpoint=""
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783760    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783763    2560 flags.go:64] FLAG: --kube-api-burst="100"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783766    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783769    2560 flags.go:64] FLAG: --kube-api-qps="50"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783771    2560 flags.go:64] FLAG: --kube-reserved=""
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783774    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783777    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783780    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783783    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783786    2560 flags.go:64] FLAG: --lock-file=""
Jul 14 15:46:56.790541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783789    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783791    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783794    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783799    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783803    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783805    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783808    2560 flags.go:64] FLAG: --logging-format="text"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783811    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783814    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783817    2560 flags.go:64] FLAG: --manifest-url=""
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783820    2560 flags.go:64] FLAG: --manifest-url-header=""
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783824    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783827    2560 flags.go:64] FLAG: --max-open-files="1000000"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783831    2560 flags.go:64] FLAG: --max-pods="110"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783834    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783836    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783839    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783842    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783845    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783849    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783852    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783859    2560 flags.go:64] FLAG: --node-status-max-images="50"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783862    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783866    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 14 15:46:56.791205 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783869    2560 flags.go:64] FLAG: --pod-cidr=""
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783872    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783878    2560 flags.go:64] FLAG: --pod-manifest-path=""
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783881    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783884    2560 flags.go:64] FLAG: --pods-per-core="0"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783888    2560 flags.go:64] FLAG: --port="10250"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783891    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783894    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cebb974023909903"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783897    2560 flags.go:64] FLAG: --qos-reserved=""
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783900    2560 flags.go:64] FLAG: --read-only-port="10255"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783903    2560 flags.go:64] FLAG: --register-node="true"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783905    2560 flags.go:64] FLAG: --register-with-taints=""
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783909    2560 flags.go:64] FLAG: --registry-burst="10"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783913    2560 flags.go:64] FLAG: --registry-qps="5"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783916    2560 flags.go:64] FLAG: --reserved-cpus=""
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783919    2560 flags.go:64] FLAG: --reserved-memory=""
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783922    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783925    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783928    2560 flags.go:64] FLAG: --rotate-certificates="false"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783931    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783934    2560 flags.go:64] FLAG: --runonce="false"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783937    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783940    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783943    2560 flags.go:64] FLAG: --seccomp-default="false"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783945    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 14 15:46:56.791879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783948    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783951    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783954    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783958    2560 flags.go:64] FLAG: --storage-driver-password="root"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783961    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783963    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783966    2560 flags.go:64] FLAG: --storage-driver-user="root"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783969    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783972    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783975    2560 flags.go:64] FLAG: --system-cgroups=""
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783978    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783983    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783987    2560 flags.go:64] FLAG: --tls-cert-file=""
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783990    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783994    2560 flags.go:64] FLAG: --tls-min-version=""
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.783997    2560 flags.go:64] FLAG: --tls-private-key-file=""
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.784000    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.784003    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.784005    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.784008    2560 flags.go:64] FLAG: --v="2"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.784012    2560 flags.go:64] FLAG: --version="false"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.784020    2560 flags.go:64] FLAG: --vmodule=""
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.784024    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.784027    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784160    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 15:46:56.792631 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784164    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784167    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784170    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784173    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784176    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784179    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784182    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784184    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784187    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784190    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784195    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784198    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784201    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784204    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784207    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784209    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784212    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784214    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784217    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784221    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 15:46:56.793327 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784223    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784225    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784228    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784230    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784233    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784237    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784240    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784244    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784247    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784250    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784252    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784255    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784257    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784260    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784262    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784265    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784267    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784270    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784272    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784275    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 15:46:56.793876 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784277    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784279    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784283    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784286    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784288    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784291    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784293    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784296    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784298    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784301    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784303    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784306    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784309    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784311    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784314    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784316    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784319    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784321    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784324    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 15:46:56.794472 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784326    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784329    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784332    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784334    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784337    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784339    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784342    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784344    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784346    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784349    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784352    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784354    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784356    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784359    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784362    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784365    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784368    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784370    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784373    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784375    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 15:46:56.794971 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784379    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784382    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784385    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784388    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784391    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784394    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784397    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784399    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784402    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784404    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784406    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784409    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784411    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784421    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784424    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784427    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784435    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784438    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 15:46:56.795463 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.784441    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.785320    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.791947    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.791963    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.791984    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.791992    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792096    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792103    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792106    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792109    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792112    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792115    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792118    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792121    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792124    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 15:46:56.795906 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792127    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792130    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792133    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792136    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792139    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792141    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792143    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792146    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792148    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792151    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792153    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792156    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792159    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792162    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792164    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792167    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792171    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792174    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792177    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792180    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 15:46:56.796284 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792184    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792186    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792189    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792192    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792195    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792198    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792201    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792203    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792206    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792209    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792211    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792214    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792216    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792219    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792221    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792224    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792226    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792229    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792231    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792233    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 15:46:56.796877 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792236    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792238    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792241    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792243    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792246    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792248    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792250    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792252    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792256    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792260    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792263    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792265    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792268    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792271    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792274    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792276    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792280    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792283    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792286    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 15:46:56.797373 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792289    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792292    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792294    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792297    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792299    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792302    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792304    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792306    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792309    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792311    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792314    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792316    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792319    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792321    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792324    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792326    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792328    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792331    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792333    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792335    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 15:46:56.797891 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792337    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792340    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792342    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792345    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792347    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792350    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792353    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792356    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792359    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792362    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792364    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.792369    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792498    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792504    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792507    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 15:46:56.798374 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792510    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792512    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792515    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792518    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792521    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792525    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792529    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792533    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792536    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792539    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792542    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792544    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792547    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792550    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792552    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792555    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792557    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792560    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792562    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 15:46:56.798751 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792564    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792567    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792569    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792572    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792574    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792577    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792580    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792583    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792585    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792588    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792590    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792593    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792595    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792597    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792600    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792602    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792605    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792607    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792610    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792612    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 15:46:56.799203 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792615    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792617    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792619    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792622    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792624    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792626    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792629    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792631    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792634    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792636    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792638    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792641    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792644    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792648    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792650    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792653    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792656    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792658    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792661    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 15:46:56.799708 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792663    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792666    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792669    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792671    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792674    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792676    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792679    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792682    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792684    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792686    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792689    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792691    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792694    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792696    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792699    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792701    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792703    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792706    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792708    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792711    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 15:46:56.800181 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792713    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792715    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792718    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792720    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792723    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792725    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792727    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792730    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792732    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792737    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792739    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792741    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792744    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792747    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792749    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792752    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792754    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 15:46:56.800729 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:56.792757    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 15:46:56.801193 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.792762    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 15:46:56.801193 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.792866    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 14 15:46:56.801193 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.794840    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 14 15:46:56.801193 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.795961    2560 server.go:1019] "Starting client certificate rotation"
Jul 14 15:46:56.801193 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.796059    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 15:46:56.801193 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.796098    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 15:46:56.801193 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.798485    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 15:46:56.803171 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.803152    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 15:46:56.817276 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.817256    2560 log.go:25] "Validated CRI v1 runtime API"
Jul 14 15:46:56.819347 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.819331    2560 log.go:25] "Validated CRI v1 image API"
Jul 14 15:46:56.821061 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.821045    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 14 15:46:56.824262 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.824232    2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8028f93c-5ed5-4aab-b87d-5db82753c3b7:/dev/nvme0n1p3 d57b6934-8d49-49f1-a4e7-cb5959fdf2a9:/dev/nvme0n1p4]
Jul 14 15:46:56.824333 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.824262    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 14 15:46:56.829269 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.829161    2560 manager.go:216] Machine: {Timestamp:2026-07-14 15:46:56.827911713 +0000 UTC m=+0.347279124 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099797 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec218f0cb76cd2924cb7e200b69f8378 SystemUUID:ec218f0c-b76c-d292-4cb7-e200b69f8378 BootID:1e168249-7ffa-47c7-8ccd-eb150f04fcfd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fd:c0:35:fc:0d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fd:c0:35:fc:0d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:e0:1f:d8:6d:fa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 14 15:46:56.829269 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.829258    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 14 15:46:56.829430 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.829386    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 14 15:46:56.830529 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.830514    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 15:46:56.832723 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.832693    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 14 15:46:56.832895 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.832725    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-20.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 14 15:46:56.834086 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.834076    2560 topology_manager.go:138] "Creating topology manager with none policy"
Jul 14 15:46:56.834122 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.834089    2560 container_manager_linux.go:309] "Creating device plugin manager"
Jul 14 15:46:56.834122 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.834096    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 15:46:56.835636 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.835625    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 15:46:56.836610 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.836600    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 14 15:46:56.839840 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.839829    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 15:46:56.839958 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.839949    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 14 15:46:56.840550 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.840539    2560 kubelet.go:486] "Attempting to sync node with API server"
Jul 14 15:46:56.840609 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.840552    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 14 15:46:56.840609 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.840569    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 14 15:46:56.840609 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.840578    2560 kubelet.go:398] "Adding apiserver pod source"
Jul 14 15:46:56.840609 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.840587    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 14 15:46:56.840898 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.840888    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 15:46:56.840938 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.840904    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 15:46:56.843088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.843074    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 14 15:46:56.844347 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.844334    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 14 15:46:56.844386 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.844365    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 14 15:46:56.844997 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.844986    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 14 15:46:56.845041 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.845008    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 14 15:46:56.845041 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.845018    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 14 15:46:56.845041 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.845024    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 14 15:46:56.845041 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.845032    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 14 15:46:56.845041 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.845038    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 14 15:46:56.845175 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.845101    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 14 15:46:56.845175 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.845108    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 14 15:46:56.845175 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.845115    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 14 15:46:56.845175 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.845121    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 14 15:46:56.845175 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.845129    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 14 15:46:56.845175 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.845137    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 14 15:46:56.846024 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.846012    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 14 15:46:56.846063 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.846026    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 14 15:46:56.849492 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.849479    2560 server.go:1268] "Started kubelet"
Jul 14 15:46:56.849601 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.849572    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 14 15:46:56.849695 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.849619    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 14 15:46:56.849745 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.849663    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 14 15:46:56.850367 ip-10-0-135-20 systemd[1]: Started Kubernetes Kubelet.
Jul 14 15:46:56.854130 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.854108    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 14 15:46:56.854586 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:56.854544    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 14 15:46:56.854666 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.854590    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-20.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 14 15:46:56.854795 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:56.854773    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 14 15:46:56.855412 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:56.855386    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 14 15:46:56.856430 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.856416    2560 server.go:310] "Adding debug handlers to kubelet server"
Jul 14 15:46:56.859577 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.859557    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 14 15:46:56.859577 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.859559    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 14 15:46:56.859692 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:56.859665    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:56.860183 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860168    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 14 15:46:56.860183 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860169    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 14 15:46:56.860329 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860195    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 14 15:46:56.860383 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860332    2560 reconstruct.go:97] "Volume reconstruction finished"
Jul 14 15:46:56.860383 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860343    2560 reconciler.go:29] "Reconciler: start to sync state"
Jul 14 15:46:56.860493 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860437    2560 factory.go:55] Registering systemd factory
Jul 14 15:46:56.860493 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860470    2560 factory.go:223] Registration of the systemd container factory successfully
Jul 14 15:46:56.860666 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860651    2560 factory.go:153] Registering CRI-O factory
Jul 14 15:46:56.860666 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860668    2560 factory.go:223] Registration of the crio container factory successfully
Jul 14 15:46:56.860861 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860727    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 14 15:46:56.860861 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860751    2560 factory.go:103] Registering Raw factory
Jul 14 15:46:56.860861 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.860768    2560 manager.go:1196] Started watching for new ooms in manager
Jul 14 15:46:56.861571 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:56.860603    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-20.ec2.internal.18c23294bff8a916  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-20.ec2.internal,UID:ip-10-0-135-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-20.ec2.internal,},FirstTimestamp:2026-07-14 15:46:56.849430806 +0000 UTC m=+0.368798218,LastTimestamp:2026-07-14 15:46:56.849430806 +0000 UTC m=+0.368798218,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-20.ec2.internal,}"
Jul 14 15:46:56.861783 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.861770    2560 manager.go:349] Starting recovery of all containers
Jul 14 15:46:56.864573 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.864552    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nxwqs"
Jul 14 15:46:56.866631 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:56.866602    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 14 15:46:56.866712 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:56.866664    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 14 15:46:56.871485 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.871309    2560 manager.go:354] Recovery completed
Jul 14 15:46:56.872876 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.872856    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nxwqs"
Jul 14 15:46:56.877131 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.877119    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 15:46:56.878947 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.878929    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 15:46:56.879020 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.878963    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 15:46:56.879020 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.878973    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID"
Jul 14 15:46:56.879426 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.879413    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 14 15:46:56.879426 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.879424    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 14 15:46:56.879535 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.879439    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 15:46:56.879535 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.879427    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 15:46:56.880158 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.880143    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 15:46:56.880211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.880169    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 15:46:56.880211 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.880187    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID"
Jul 14 15:46:56.880643 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.880536    2560 policy_none.go:49] "None policy: Start"
Jul 14 15:46:56.880643 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.880549    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 14 15:46:56.880643 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.880560    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 14 15:46:56.881563 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.881549    2560 policy_none.go:47] "Start"
Jul 14 15:46:56.881630 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:56.881483    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-20.ec2.internal.18c23294c1bb061e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-20.ec2.internal,UID:ip-10-0-135-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-20.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-20.ec2.internal,},FirstTimestamp:2026-07-14 15:46:56.878945822 +0000 UTC m=+0.398313234,LastTimestamp:2026-07-14 15:46:56.878945822 +0000 UTC m=+0.398313234,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-20.ec2.internal,}"
Jul 14 15:46:56.912956 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.912939    2560 manager.go:337] "Starting Device Plugin manager"
Jul 14 15:46:56.913044 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:56.912968    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 14 15:46:56.913044 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.912977    2560 server.go:87] "Starting device plugin registration server"
Jul 14 15:46:56.913216 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.913204    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 14 15:46:56.913254 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.913219    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 14 15:46:56.913306 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.913291    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 14 15:46:56.913379 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.913370    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 14 15:46:56.913424 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:56.913380    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 14 15:46:56.913810 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:56.913791    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 14 15:46:56.913898 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:56.913834    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:57.013834 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.013804    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 15:46:57.014921 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.014689    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 15:46:57.014921 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.014715    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 15:46:57.014921 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.014725    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID"
Jul 14 15:46:57.014921 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.014749    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.024045 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.024002    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.024045 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.024023    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-20.ec2.internal\": node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:57.024146 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.024129    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 14 15:46:57.025479 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.025438    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 14 15:46:57.025522 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.025502    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 14 15:46:57.025558 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.025525    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 14 15:46:57.025606 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.025561    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 14 15:46:57.027825 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.027806    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 15:46:57.070975 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.070952    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:57.126165 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.126136    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal"]
Jul 14 15:46:57.126260 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.126226    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 15:46:57.127858 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.127841    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 15:46:57.127970 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.127877    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 15:46:57.127970 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.127892    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID"
Jul 14 15:46:57.129140 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.129124    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 15:46:57.129250 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.129234    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.129301 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.129265    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 15:46:57.129916 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.129887    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 15:46:57.129916 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.129913    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 15:46:57.129916 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.129917    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 15:46:57.130086 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.129927    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID"
Jul 14 15:46:57.130086 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.129934    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 15:46:57.130086 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.129946    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID"
Jul 14 15:46:57.131175 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.131159    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.131267 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.131184    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 15:46:57.131853 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.131839    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 15:46:57.131853 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.131862    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 15:46:57.131966 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.131877    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeHasSufficientPID"
Jul 14 15:46:57.161682 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.161659    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0009cb587a3b9c89871f7a780a349638-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"0009cb587a3b9c89871f7a780a349638\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.161767 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.161698    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0009cb587a3b9c89871f7a780a349638-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"0009cb587a3b9c89871f7a780a349638\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.161767 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.161717    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b65546031303e49503806457ed26462-config\") pod \"kube-apiserver-proxy-ip-10-0-135-20.ec2.internal\" (UID: \"7b65546031303e49503806457ed26462\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.164055 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.164036    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-20.ec2.internal\" not found" node="ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.168275 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.168260    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-20.ec2.internal\" not found" node="ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.171312 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.171297    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:57.262314 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.262292    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0009cb587a3b9c89871f7a780a349638-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"0009cb587a3b9c89871f7a780a349638\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.262420 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.262325    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0009cb587a3b9c89871f7a780a349638-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"0009cb587a3b9c89871f7a780a349638\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.262420 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.262348    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b65546031303e49503806457ed26462-config\") pod \"kube-apiserver-proxy-ip-10-0-135-20.ec2.internal\" (UID: \"7b65546031303e49503806457ed26462\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.262420 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.262398    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0009cb587a3b9c89871f7a780a349638-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"0009cb587a3b9c89871f7a780a349638\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.262572 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.262430    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b65546031303e49503806457ed26462-config\") pod \"kube-apiserver-proxy-ip-10-0-135-20.ec2.internal\" (UID: \"7b65546031303e49503806457ed26462\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.262572 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.262400    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0009cb587a3b9c89871f7a780a349638-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal\" (UID: \"0009cb587a3b9c89871f7a780a349638\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.271630 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.271612    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:57.372509 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.372424    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:57.466930 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.466905    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.471476 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.470963    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.473197 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.473182    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:57.573885 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.573858    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:57.674486 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.674409    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:57.710745 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.710722    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 15:46:57.774689 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.774660    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:57.796004 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.795980    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 14 15:46:57.796111 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.796093    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 15:46:57.796153 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.796134    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 15:46:57.874731 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:57.874708    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-20.ec2.internal\" not found"
Jul 14 15:46:57.877158 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.877133    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 15:41:57 +0000 UTC" deadline="2028-02-02 16:04:55.666966999 +0000 UTC"
Jul 14 15:46:57.877209 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.877158    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13632h17m57.789811684s"
Jul 14 15:46:57.944832 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.944814    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 15:46:57.960189 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.960167    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.972145 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.972121    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 15:46:57.974259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.974233    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal"
Jul 14 15:46:57.986290 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:57.986272    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 15:46:58.028867 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:58.028840    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0009cb587a3b9c89871f7a780a349638.slice/crio-3e9c296a90ab888f45b94815935ee309a6271a3f56e319bceeab87e9df05f093 WatchSource:0}: Error finding container 3e9c296a90ab888f45b94815935ee309a6271a3f56e319bceeab87e9df05f093: Status 404 returned error can't find the container with id 3e9c296a90ab888f45b94815935ee309a6271a3f56e319bceeab87e9df05f093
Jul 14 15:46:58.034094 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.034078    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 15:46:58.175650 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.175625    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 15:46:58.844398 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.844355    2560 apiserver.go:52] "Watching apiserver"
Jul 14 15:46:58.853151 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.853129    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 14 15:46:58.855635 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.855612    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-lx86m","kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb","openshift-cluster-node-tuning-operator/tuned-lmv5z","openshift-multus/multus-additional-cni-plugins-g9l8z","openshift-multus/multus-h8lpb","openshift-network-diagnostics/network-check-target-jxlsn","openshift-network-operator/iptables-alerter-98fqw","kube-system/global-pull-secret-syncer-8674z","openshift-dns/node-resolver-mtpnp","openshift-image-registry/node-ca-sbkld","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal","openshift-multus/network-metrics-daemon-w92wj","openshift-ovn-kubernetes/ovnkube-node-8c45s"]
Jul 14 15:46:58.857885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.857871    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:46:58.859030 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.859010    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:46:58.859162 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:58.859083    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:46:58.860111 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.860089    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sbkld"
Jul 14 15:46:58.860195 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.860163    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 14 15:46:58.860997 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.860980    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 14 15:46:58.861093 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.860983    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 14 15:46:58.861093 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.860984    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rnnwp\""
Jul 14 15:46:58.861327 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.861311    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-98fqw"
Jul 14 15:46:58.862480 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.862446    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:58.863477 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.863459    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r6fxk\""
Jul 14 15:46:58.863801 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.863777    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.864014 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.863995    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 14 15:46:58.864318 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.864296    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 14 15:46:58.864951 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.864934    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:46:58.865330 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:58.865161    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:46:58.865330 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.865171    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 14 15:46:58.865330 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.865231    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 14 15:46:58.865682 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.865659    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 14 15:46:58.865971 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.865948    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 14 15:46:58.866071 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.865990    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 14 15:46:58.866071 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.865993    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s8qgb\""
Jul 14 15:46:58.866071 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.866038    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-btfwv\""
Jul 14 15:46:58.866538 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.866503    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.866736 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.866715    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 14 15:46:58.867235 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.867218    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5dl98\""
Jul 14 15:46:58.867235 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.867227    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 14 15:46:58.867390 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.867260    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 14 15:46:58.867390 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.867227    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 14 15:46:58.867712 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.867697    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:46:58.867769 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:58.867752    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:46:58.869045 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.869026    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mtpnp"
Jul 14 15:46:58.869799 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.869782    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 14 15:46:58.870054 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.870033    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99983123-1bb8-4031-990a-bf3bfc8f5cda-iptables-alerter-script\") pod \"iptables-alerter-98fqw\" (UID: \"99983123-1bb8-4031-990a-bf3bfc8f5cda\") " pod="openshift-network-operator/iptables-alerter-98fqw"
Jul 14 15:46:58.870155 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.870065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99983123-1bb8-4031-990a-bf3bfc8f5cda-host-slash\") pod \"iptables-alerter-98fqw\" (UID: \"99983123-1bb8-4031-990a-bf3bfc8f5cda\") " pod="openshift-network-operator/iptables-alerter-98fqw"
Jul 14 15:46:58.870155 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.870088    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr7xr\" (UniqueName: \"kubernetes.io/projected/99983123-1bb8-4031-990a-bf3bfc8f5cda-kube-api-access-gr7xr\") pod \"iptables-alerter-98fqw\" (UID: \"99983123-1bb8-4031-990a-bf3bfc8f5cda\") " pod="openshift-network-operator/iptables-alerter-98fqw"
Jul 14 15:46:58.870155 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.870116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f5ac6baa-690d-477c-8090-199341500f68-agent-certs\") pod \"konnectivity-agent-lx86m\" (UID: \"f5ac6baa-690d-477c-8090-199341500f68\") " pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:46:58.870300 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.870178    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d2g2\" (UniqueName: \"kubernetes.io/projected/9bbccbea-f177-4169-a89d-a09d860543b0-kube-api-access-9d2g2\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:46:58.870300 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.870214    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4921108-78c3-47e5-9965-2882382685de-host\") pod \"node-ca-sbkld\" (UID: \"c4921108-78c3-47e5-9965-2882382685de\") " pod="openshift-image-registry/node-ca-sbkld"
Jul 14 15:46:58.870300 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.870261    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4921108-78c3-47e5-9965-2882382685de-serviceca\") pod \"node-ca-sbkld\" (UID: \"c4921108-78c3-47e5-9965-2882382685de\") " pod="openshift-image-registry/node-ca-sbkld"
Jul 14 15:46:58.870300 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.870283    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f5ac6baa-690d-477c-8090-199341500f68-konnectivity-ca\") pod \"konnectivity-agent-lx86m\" (UID: \"f5ac6baa-690d-477c-8090-199341500f68\") " pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:46:58.870300 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.870300    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:46:58.870514 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.870320    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqw9m\" (UniqueName: \"kubernetes.io/projected/c4921108-78c3-47e5-9965-2882382685de-kube-api-access-rqw9m\") pod \"node-ca-sbkld\" (UID: \"c4921108-78c3-47e5-9965-2882382685de\") " pod="openshift-image-registry/node-ca-sbkld"
Jul 14 15:46:58.870726 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.870708    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:58.871025 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.871006    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 14 15:46:58.871152 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.871134    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 14 15:46:58.871219 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.871142    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jlpxs\""
Jul 14 15:46:58.871491 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.871478    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 14 15:46:58.872358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.872336    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.873558 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.873456    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9sd4n\""
Jul 14 15:46:58.873762 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.873739    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 14 15:46:58.874507 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.874480    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 14 15:46:58.874665 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.874601    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 14 15:46:58.875662 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.875641    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 14 15:46:58.876313 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.876288    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ddl8k\""
Jul 14 15:46:58.876400 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.876381    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 14 15:46:58.876511 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.876486    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 14 15:46:58.877028 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.877010    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 14 15:46:58.877485 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.877030    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 14 15:46:58.877577 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.877514    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fwqcx\""
Jul 14 15:46:58.877577 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.877067    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 14 15:46:58.877577 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.877067    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 15:46:58.877718 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.877092    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 14 15:46:58.902898 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.902881    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nkrvq"
Jul 14 15:46:58.913688 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.913674    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nkrvq"
Jul 14 15:46:58.961771 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.961753    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 14 15:46:58.971159 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971139    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dmk4\" (UniqueName: \"kubernetes.io/projected/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-kube-api-access-5dmk4\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:58.971250 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971164    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-slash\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.971250 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971188    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-etc-openvswitch\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.971250 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971209    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99983123-1bb8-4031-990a-bf3bfc8f5cda-iptables-alerter-script\") pod \"iptables-alerter-98fqw\" (UID: \"99983123-1bb8-4031-990a-bf3bfc8f5cda\") " pod="openshift-network-operator/iptables-alerter-98fqw"
Jul 14 15:46:58.971250 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971236    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwbkj\" (UniqueName: \"kubernetes.io/projected/77a7601c-9b76-4736-94a3-0f9007c6c221-kube-api-access-pwbkj\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:58.971421 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971261    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-device-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:58.971421 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971284    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-system-cni-dir\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.971421 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971319    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/77a7601c-9b76-4736-94a3-0f9007c6c221-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:58.971421 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971346    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-kubernetes\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.971421 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971370    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-host\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.971421 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-run-openvswitch\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.971691 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-ovnkube-script-lib\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.971691 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971496    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:46:58.971691 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971532    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-kubelet\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.971691 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:58.971629    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:46:58.971691 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971645    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4921108-78c3-47e5-9965-2882382685de-host\") pod \"node-ca-sbkld\" (UID: \"c4921108-78c3-47e5-9965-2882382685de\") " pod="openshift-image-registry/node-ca-sbkld"
Jul 14 15:46:58.971691 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971677    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4921108-78c3-47e5-9965-2882382685de-serviceca\") pod \"node-ca-sbkld\" (UID: \"c4921108-78c3-47e5-9965-2882382685de\") " pod="openshift-image-registry/node-ca-sbkld"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971698    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hxj6\" (UniqueName: \"kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6\") pod \"network-check-target-jxlsn\" (UID: \"72ca74e2-c3b6-42e4-98f2-f12330a7bb48\") " pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:58.971715    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs podName:9bbccbea-f177-4169-a89d-a09d860543b0 nodeName:}" failed. No retries permitted until 2026-07-14 15:46:59.471695165 +0000 UTC m=+2.991062569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs") pod "network-metrics-daemon-w92wj" (UID: "9bbccbea-f177-4169-a89d-a09d860543b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971744    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-run-ovn-kubernetes\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971750    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4921108-78c3-47e5-9965-2882382685de-host\") pod \"node-ca-sbkld\" (UID: \"c4921108-78c3-47e5-9965-2882382685de\") " pod="openshift-image-registry/node-ca-sbkld"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971780    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99983123-1bb8-4031-990a-bf3bfc8f5cda-iptables-alerter-script\") pod \"iptables-alerter-98fqw\" (UID: \"99983123-1bb8-4031-990a-bf3bfc8f5cda\") " pod="openshift-network-operator/iptables-alerter-98fqw"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971790    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f5ac6baa-690d-477c-8090-199341500f68-konnectivity-ca\") pod \"konnectivity-agent-lx86m\" (UID: \"f5ac6baa-690d-477c-8090-199341500f68\") " pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971818    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqw9m\" (UniqueName: \"kubernetes.io/projected/c4921108-78c3-47e5-9965-2882382685de-kube-api-access-rqw9m\") pod \"node-ca-sbkld\" (UID: \"c4921108-78c3-47e5-9965-2882382685de\") " pod="openshift-image-registry/node-ca-sbkld"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971847    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-cni-binary-copy\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971873    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-socket-dir-parent\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971902    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-run-netns\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971927    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zmlr\" (UniqueName: \"kubernetes.io/projected/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-kube-api-access-7zmlr\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971951    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:46:58.971984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.971977    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-cni-dir\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972003    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-system-cni-dir\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972029    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-os-release\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972056    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-modprobe-d\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-sysconfig\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972118    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-var-lib-kubelet\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972143    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-sysctl-conf\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972158    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-run\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972121    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4921108-78c3-47e5-9965-2882382685de-serviceca\") pod \"node-ca-sbkld\" (UID: \"c4921108-78c3-47e5-9965-2882382685de\") " pod="openshift-image-registry/node-ca-sbkld"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972175    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99983123-1bb8-4031-990a-bf3bfc8f5cda-host-slash\") pod \"iptables-alerter-98fqw\" (UID: \"99983123-1bb8-4031-990a-bf3bfc8f5cda\") " pod="openshift-network-operator/iptables-alerter-98fqw"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972198    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/77a7601c-9b76-4736-94a3-0f9007c6c221-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972218    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99983123-1bb8-4031-990a-bf3bfc8f5cda-host-slash\") pod \"iptables-alerter-98fqw\" (UID: \"99983123-1bb8-4031-990a-bf3bfc8f5cda\") " pod="openshift-network-operator/iptables-alerter-98fqw"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972226    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjfmv\" (UniqueName: \"kubernetes.io/projected/17d11734-7292-48a0-b53c-b3034fd7ae90-kube-api-access-zjfmv\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972251    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17d11734-7292-48a0-b53c-b3034fd7ae90-tmp\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972275    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-cni-bin\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972301    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.972629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972349    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-ovnkube-config\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972384    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-env-overrides\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972414    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d2g2\" (UniqueName: \"kubernetes.io/projected/9bbccbea-f177-4169-a89d-a09d860543b0-kube-api-access-9d2g2\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972445    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b801942-157e-4913-98a7-44d9440b3257-dbus\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972492    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972535    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cee7827-8938-4376-8ba2-65a519b89996-tmp-dir\") pod \"node-resolver-mtpnp\" (UID: \"0cee7827-8938-4376-8ba2-65a519b89996\") " pod="openshift-dns/node-resolver-mtpnp"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972561    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972585    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-registration-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972605    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-systemd-units\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972621    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-run-systemd\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972647    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-var-lib-cni-multus\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-etc-kubernetes\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972705    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-cnibin\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77a7601c-9b76-4736-94a3-0f9007c6c221-cni-binary-copy\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972743    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-lib-modules\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-run-netns\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972778    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-ovn-node-metrics-cert\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972807    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f5ac6baa-690d-477c-8090-199341500f68-agent-certs\") pod \"konnectivity-agent-lx86m\" (UID: \"f5ac6baa-690d-477c-8090-199341500f68\") " pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972837    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-socket-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972845    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f5ac6baa-690d-477c-8090-199341500f68-konnectivity-ca\") pod \"konnectivity-agent-lx86m\" (UID: \"f5ac6baa-690d-477c-8090-199341500f68\") " pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972867    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-var-lib-openvswitch\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972891    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-cni-netd\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972920    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr7xr\" (UniqueName: \"kubernetes.io/projected/99983123-1bb8-4031-990a-bf3bfc8f5cda-kube-api-access-gr7xr\") pod \"iptables-alerter-98fqw\" (UID: \"99983123-1bb8-4031-990a-bf3bfc8f5cda\") " pod="openshift-network-operator/iptables-alerter-98fqw"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972948    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-var-lib-cni-bin\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.972992    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0cee7827-8938-4376-8ba2-65a519b89996-hosts-file\") pod \"node-resolver-mtpnp\" (UID: \"0cee7827-8938-4376-8ba2-65a519b89996\") " pod="openshift-dns/node-resolver-mtpnp"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973022    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-run-ovn\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973045    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-node-log\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973061    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973086    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-log-socket\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973126    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msjml\" (UniqueName: \"kubernetes.io/projected/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-kube-api-access-msjml\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973168    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-os-release\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973194    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-var-lib-kubelet\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973227    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-hostroot\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.973941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973261    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-daemon-config\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973276    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b801942-157e-4913-98a7-44d9440b3257-kubelet-config\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973298    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973314    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-sysctl-d\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973343    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-tuned\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973377    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-conf-dir\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-sys\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-cnibin\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973434    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-run-k8s-cni-cncf-io\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973478    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-run-multus-certs\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973499    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-systemd\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973518    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8ktq\" (UniqueName: \"kubernetes.io/projected/0cee7827-8938-4376-8ba2-65a519b89996-kube-api-access-c8ktq\") pod \"node-resolver-mtpnp\" (UID: \"0cee7827-8938-4376-8ba2-65a519b89996\") " pod="openshift-dns/node-resolver-mtpnp"
Jul 14 15:46:58.974443 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.973541    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-sys-fs\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:58.976272 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.976251    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f5ac6baa-690d-477c-8090-199341500f68-agent-certs\") pod \"konnectivity-agent-lx86m\" (UID: \"f5ac6baa-690d-477c-8090-199341500f68\") " pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:46:58.984597 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.984569    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqw9m\" (UniqueName: \"kubernetes.io/projected/c4921108-78c3-47e5-9965-2882382685de-kube-api-access-rqw9m\") pod \"node-ca-sbkld\" (UID: \"c4921108-78c3-47e5-9965-2882382685de\") " pod="openshift-image-registry/node-ca-sbkld"
Jul 14 15:46:58.986999 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.986977    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d2g2\" (UniqueName: \"kubernetes.io/projected/9bbccbea-f177-4169-a89d-a09d860543b0-kube-api-access-9d2g2\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:46:58.987105 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:58.987085    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr7xr\" (UniqueName: \"kubernetes.io/projected/99983123-1bb8-4031-990a-bf3bfc8f5cda-kube-api-access-gr7xr\") pod \"iptables-alerter-98fqw\" (UID: \"99983123-1bb8-4031-990a-bf3bfc8f5cda\") " pod="openshift-network-operator/iptables-alerter-98fqw"
Jul 14 15:46:59.029326 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.029272    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" event={"ID":"0009cb587a3b9c89871f7a780a349638","Type":"ContainerStarted","Data":"3e9c296a90ab888f45b94815935ee309a6271a3f56e319bceeab87e9df05f093"}
Jul 14 15:46:59.074277 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074255    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-sys\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.074410 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074285    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-cnibin\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.074410 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074301    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-run-k8s-cni-cncf-io\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.074410 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074326    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-run-multus-certs\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.074410 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074343    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-systemd\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.074410 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074359    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8ktq\" (UniqueName: \"kubernetes.io/projected/0cee7827-8938-4376-8ba2-65a519b89996-kube-api-access-c8ktq\") pod \"node-resolver-mtpnp\" (UID: \"0cee7827-8938-4376-8ba2-65a519b89996\") " pod="openshift-dns/node-resolver-mtpnp"
Jul 14 15:46:59.074410 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074374    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-sys-fs\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.074410 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074395    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dmk4\" (UniqueName: \"kubernetes.io/projected/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-kube-api-access-5dmk4\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.074410 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074394    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-sys\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074408    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-run-k8s-cni-cncf-io\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074414    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-run-multus-certs\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074438    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-slash\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074448    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-systemd\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074475    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-sys-fs\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074494    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-slash\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074497    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-cnibin\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074495    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-etc-openvswitch\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074528    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-etc-openvswitch\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074558    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwbkj\" (UniqueName: \"kubernetes.io/projected/77a7601c-9b76-4736-94a3-0f9007c6c221-kube-api-access-pwbkj\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074585    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-device-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074610    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-system-cni-dir\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074637    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/77a7601c-9b76-4736-94a3-0f9007c6c221-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074666    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-kubernetes\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074682    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-system-cni-dir\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074686    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-device-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074690    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-host\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.074871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074732    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-host\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074734    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-run-openvswitch\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074731    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-kubernetes\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-run-openvswitch\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074768    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-ovnkube-script-lib\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074809    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-kubelet\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074836    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxj6\" (UniqueName: \"kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6\") pod \"network-check-target-jxlsn\" (UID: \"72ca74e2-c3b6-42e4-98f2-f12330a7bb48\") " pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074862    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-run-ovn-kubernetes\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074863    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-kubelet\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074889    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-cni-binary-copy\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074915    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-socket-dir-parent\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074926    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-run-ovn-kubernetes\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074940    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-run-netns\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zmlr\" (UniqueName: \"kubernetes.io/projected/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-kube-api-access-7zmlr\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.074995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075005    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-socket-dir-parent\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075021    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-cni-dir\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.075689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075022    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-run-netns\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075059    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-system-cni-dir\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075089    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-os-release\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.075105    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075118    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-modprobe-d\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075142    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-sysconfig\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075162    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-system-cni-dir\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.075169    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret podName:0b801942-157e-4913-98a7-44d9440b3257 nodeName:}" failed. No retries permitted until 2026-07-14 15:46:59.575148639 +0000 UTC m=+3.094516044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret") pod "global-pull-secret-syncer-8674z" (UID: "0b801942-157e-4913-98a7-44d9440b3257") : object "kube-system"/"original-pull-secret" not registered
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075179    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-cni-dir\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075184    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-sysconfig\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075216    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-os-release\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075230    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-var-lib-kubelet\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075258    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-var-lib-kubelet\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075259    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-sysctl-conf\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075283    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-modprobe-d\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075288    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-run\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075325    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/77a7601c-9b76-4736-94a3-0f9007c6c221-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.076586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075339    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-run\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075355    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjfmv\" (UniqueName: \"kubernetes.io/projected/17d11734-7292-48a0-b53c-b3034fd7ae90-kube-api-access-zjfmv\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075380    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17d11734-7292-48a0-b53c-b3034fd7ae90-tmp\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075397    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-sysctl-conf\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075404    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-cni-bin\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075443    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-cni-bin\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075441    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075497    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075481    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-ovnkube-config\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075509    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-cni-binary-copy\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075533    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-env-overrides\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075558    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b801942-157e-4913-98a7-44d9440b3257-dbus\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075583    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075607    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cee7827-8938-4376-8ba2-65a519b89996-tmp-dir\") pod \"node-resolver-mtpnp\" (UID: \"0cee7827-8938-4376-8ba2-65a519b89996\") " pod="openshift-dns/node-resolver-mtpnp"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075633    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075660    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-registration-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075683    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-systemd-units\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077259 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075706    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-run-systemd\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075725    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b801942-157e-4913-98a7-44d9440b3257-dbus\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075738    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-var-lib-cni-multus\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075765    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-etc-kubernetes\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075785    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-registration-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075791    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-cnibin\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075795    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075815    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77a7601c-9b76-4736-94a3-0f9007c6c221-cni-binary-copy\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075844    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-lib-modules\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075870    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-run-netns\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075895    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-ovn-node-metrics-cert\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075914    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/77a7601c-9b76-4736-94a3-0f9007c6c221-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075925    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-socket-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075949    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-var-lib-openvswitch\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075971    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-cni-netd\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075976    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075981    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-ovnkube-script-lib\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.077843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.075996    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-var-lib-cni-bin\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0cee7827-8938-4376-8ba2-65a519b89996-hosts-file\") pod \"node-resolver-mtpnp\" (UID: \"0cee7827-8938-4376-8ba2-65a519b89996\") " pod="openshift-dns/node-resolver-mtpnp"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076027    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-env-overrides\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076036    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77a7601c-9b76-4736-94a3-0f9007c6c221-cnibin\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076037    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-var-lib-cni-multus\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-run-ovn\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076059    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-ovnkube-config\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076072    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-systemd-units\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076080    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-node-log\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076108    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-run-ovn\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076112    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cee7827-8938-4376-8ba2-65a519b89996-tmp-dir\") pod \"node-resolver-mtpnp\" (UID: \"0cee7827-8938-4376-8ba2-65a519b89996\") " pod="openshift-dns/node-resolver-mtpnp"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076125    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-node-log\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-var-lib-openvswitch\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076114    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/77a7601c-9b76-4736-94a3-0f9007c6c221-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076149    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-log-socket\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076153    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-var-lib-cni-bin\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076161    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-lib-modules\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076172    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-etc-kubernetes\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.078611 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076178    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msjml\" (UniqueName: \"kubernetes.io/projected/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-kube-api-access-msjml\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076182    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-cni-netd\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076223    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-log-socket\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076241    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0cee7827-8938-4376-8ba2-65a519b89996-hosts-file\") pod \"node-resolver-mtpnp\" (UID: \"0cee7827-8938-4376-8ba2-65a519b89996\") " pod="openshift-dns/node-resolver-mtpnp"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076243    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-run-systemd\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076249    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-os-release\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076249    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-socket-dir\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076273    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-host-run-netns\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076283    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-var-lib-kubelet\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076300    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-os-release\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076311    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-hostroot\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076336    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-host-var-lib-kubelet\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076339    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-daemon-config\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076373    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b801942-157e-4913-98a7-44d9440b3257-kubelet-config\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076381    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-hostroot\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076432    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b801942-157e-4913-98a7-44d9440b3257-kubelet-config\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076475    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076487    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77a7601c-9b76-4736-94a3-0f9007c6c221-cni-binary-copy\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.079633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-sysctl-d\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.080585 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076568    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-tuned\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.080585 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076593    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-conf-dir\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.080585 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076625    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-sysctl-d\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.080585 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076626    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.080585 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.076659    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-conf-dir\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.080585 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.077121    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-multus-daemon-config\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.080585 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.078411    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17d11734-7292-48a0-b53c-b3034fd7ae90-tmp\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.080585 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.078573    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-ovn-node-metrics-cert\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.080585 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.078877    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17d11734-7292-48a0-b53c-b3034fd7ae90-etc-tuned\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.082130 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.082101    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 15:46:59.091160 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.091136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dmk4\" (UniqueName: \"kubernetes.io/projected/631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca-kube-api-access-5dmk4\") pod \"aws-ebs-csi-driver-node-qh2fb\" (UID: \"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.098131 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.098081    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8ktq\" (UniqueName: \"kubernetes.io/projected/0cee7827-8938-4376-8ba2-65a519b89996-kube-api-access-c8ktq\") pod \"node-resolver-mtpnp\" (UID: \"0cee7827-8938-4376-8ba2-65a519b89996\") " pod="openshift-dns/node-resolver-mtpnp"
Jul 14 15:46:59.100584 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.100501    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwbkj\" (UniqueName: \"kubernetes.io/projected/77a7601c-9b76-4736-94a3-0f9007c6c221-kube-api-access-pwbkj\") pod \"multus-additional-cni-plugins-g9l8z\" (UID: \"77a7601c-9b76-4736-94a3-0f9007c6c221\") " pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.100712 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.100696    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msjml\" (UniqueName: \"kubernetes.io/projected/ff82c9a1-cbcf-485b-9fe0-6e24d41da713-kube-api-access-msjml\") pod \"ovnkube-node-8c45s\" (UID: \"ff82c9a1-cbcf-485b-9fe0-6e24d41da713\") " pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.100761 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.100741    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zmlr\" (UniqueName: \"kubernetes.io/projected/7b9dd8f7-527c-4dab-a52b-17c38f30fad1-kube-api-access-7zmlr\") pod \"multus-h8lpb\" (UID: \"7b9dd8f7-527c-4dab-a52b-17c38f30fad1\") " pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.101409 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.101382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjfmv\" (UniqueName: \"kubernetes.io/projected/17d11734-7292-48a0-b53c-b3034fd7ae90-kube-api-access-zjfmv\") pod \"tuned-lmv5z\" (UID: \"17d11734-7292-48a0-b53c-b3034fd7ae90\") " pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.102193 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.102179    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 15:46:59.102252 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.102197    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 15:46:59.102252 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.102207    2560 projected.go:196] Error preparing data for projected volume kube-api-access-5hxj6 for pod openshift-network-diagnostics/network-check-target-jxlsn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:46:59.102314 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.102276    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6 podName:72ca74e2-c3b6-42e4-98f2-f12330a7bb48 nodeName:}" failed. No retries permitted until 2026-07-14 15:46:59.602251249 +0000 UTC m=+3.121618670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5hxj6" (UniqueName: "kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6") pod "network-check-target-jxlsn" (UID: "72ca74e2-c3b6-42e4-98f2-f12330a7bb48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:46:59.160270 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:59.160246    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b65546031303e49503806457ed26462.slice/crio-d34007e209d501f231f03f0cdfec00fdc80e4536fdcf77f6d93036ff06d777d0 WatchSource:0}: Error finding container d34007e209d501f231f03f0cdfec00fdc80e4536fdcf77f6d93036ff06d777d0: Status 404 returned error can't find the container with id d34007e209d501f231f03f0cdfec00fdc80e4536fdcf77f6d93036ff06d777d0
Jul 14 15:46:59.169410 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.169385    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:46:59.177108 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:59.177089    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ac6baa_690d_477c_8090_199341500f68.slice/crio-849eac5ea5c68ae9abe02bc1f42da64bf86610b80c4becc6878631c2d3ffc376 WatchSource:0}: Error finding container 849eac5ea5c68ae9abe02bc1f42da64bf86610b80c4becc6878631c2d3ffc376: Status 404 returned error can't find the container with id 849eac5ea5c68ae9abe02bc1f42da64bf86610b80c4becc6878631c2d3ffc376
Jul 14 15:46:59.177992 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.177977    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sbkld"
Jul 14 15:46:59.184934 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:59.184913    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4921108_78c3_47e5_9965_2882382685de.slice/crio-783f71fd78e3b2daf85645b3ab44f371dd219bbf820134fa87be280f3cd3cbd9 WatchSource:0}: Error finding container 783f71fd78e3b2daf85645b3ab44f371dd219bbf820134fa87be280f3cd3cbd9: Status 404 returned error can't find the container with id 783f71fd78e3b2daf85645b3ab44f371dd219bbf820134fa87be280f3cd3cbd9
Jul 14 15:46:59.186067 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.186048    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-98fqw"
Jul 14 15:46:59.191212 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.191196    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g9l8z"
Jul 14 15:46:59.194539 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:59.194512    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99983123_1bb8_4031_990a_bf3bfc8f5cda.slice/crio-72bc72d447c7910f7b645fbb691533aac3ab9c392436df4fa2b46a7950dc11c2 WatchSource:0}: Error finding container 72bc72d447c7910f7b645fbb691533aac3ab9c392436df4fa2b46a7950dc11c2: Status 404 returned error can't find the container with id 72bc72d447c7910f7b645fbb691533aac3ab9c392436df4fa2b46a7950dc11c2
Jul 14 15:46:59.195946 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.195717    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h8lpb"
Jul 14 15:46:59.201581 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.201554    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lmv5z"
Jul 14 15:46:59.202568 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:59.202545    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a7601c_9b76_4736_94a3_0f9007c6c221.slice/crio-a8fe8d72965720adf17162df80cb3f0f0282a435b9b55c7534dcfbc1e1733bab WatchSource:0}: Error finding container a8fe8d72965720adf17162df80cb3f0f0282a435b9b55c7534dcfbc1e1733bab: Status 404 returned error can't find the container with id a8fe8d72965720adf17162df80cb3f0f0282a435b9b55c7534dcfbc1e1733bab
Jul 14 15:46:59.203830 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:59.203806    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b9dd8f7_527c_4dab_a52b_17c38f30fad1.slice/crio-c9f1d445dbe084b96fe43a7ed4817ef5a4c9e4e3cfa0f4f0e0bb1e6fca420f78 WatchSource:0}: Error finding container c9f1d445dbe084b96fe43a7ed4817ef5a4c9e4e3cfa0f4f0e0bb1e6fca420f78: Status 404 returned error can't find the container with id c9f1d445dbe084b96fe43a7ed4817ef5a4c9e4e3cfa0f4f0e0bb1e6fca420f78
Jul 14 15:46:59.208138 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.208122    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mtpnp"
Jul 14 15:46:59.209653 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:59.209630    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17d11734_7292_48a0_b53c_b3034fd7ae90.slice/crio-05d024df0010f5647203d8a38e3b740496f988fca1bca988f75ce02f1e4f2138 WatchSource:0}: Error finding container 05d024df0010f5647203d8a38e3b740496f988fca1bca988f75ce02f1e4f2138: Status 404 returned error can't find the container with id 05d024df0010f5647203d8a38e3b740496f988fca1bca988f75ce02f1e4f2138
Jul 14 15:46:59.213488 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.213424    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb"
Jul 14 15:46:59.216250 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:59.216124    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cee7827_8938_4376_8ba2_65a519b89996.slice/crio-395baced0ec0ceda38460dcad22ea818d09675157cc9b20bb3567432193c32bb WatchSource:0}: Error finding container 395baced0ec0ceda38460dcad22ea818d09675157cc9b20bb3567432193c32bb: Status 404 returned error can't find the container with id 395baced0ec0ceda38460dcad22ea818d09675157cc9b20bb3567432193c32bb
Jul 14 15:46:59.221212 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.221193    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:46:59.222842 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:59.222822    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631b26cf_cb79_47a0_9e44_dcf2f5a1d0ca.slice/crio-51b5d0c6366e446efdb4a91f5f542804c1e01ff6bb3d2c549a1ae366ae08483e WatchSource:0}: Error finding container 51b5d0c6366e446efdb4a91f5f542804c1e01ff6bb3d2c549a1ae366ae08483e: Status 404 returned error can't find the container with id 51b5d0c6366e446efdb4a91f5f542804c1e01ff6bb3d2c549a1ae366ae08483e
Jul 14 15:46:59.229004 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:46:59.228986    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff82c9a1_cbcf_485b_9fe0_6e24d41da713.slice/crio-66c8a0472c87b83d4442b1f0f6b22386296bb94907b3e36c0c3dd6d3e9a23cdb WatchSource:0}: Error finding container 66c8a0472c87b83d4442b1f0f6b22386296bb94907b3e36c0c3dd6d3e9a23cdb: Status 404 returned error can't find the container with id 66c8a0472c87b83d4442b1f0f6b22386296bb94907b3e36c0c3dd6d3e9a23cdb
Jul 14 15:46:59.479684 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.479656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:46:59.479815 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.479787    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:46:59.479870 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.479840    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs podName:9bbccbea-f177-4169-a89d-a09d860543b0 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:00.479827018 +0000 UTC m=+3.999194419 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs") pod "network-metrics-daemon-w92wj" (UID: "9bbccbea-f177-4169-a89d-a09d860543b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:46:59.581044 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.581008    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:46:59.581216 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.581178    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 15:46:59.581258 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.581252    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret podName:0b801942-157e-4913-98a7-44d9440b3257 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:00.581222905 +0000 UTC m=+4.100590307 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret") pod "global-pull-secret-syncer-8674z" (UID: "0b801942-157e-4913-98a7-44d9440b3257") : object "kube-system"/"original-pull-secret" not registered
Jul 14 15:46:59.681994 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.681910    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxj6\" (UniqueName: \"kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6\") pod \"network-check-target-jxlsn\" (UID: \"72ca74e2-c3b6-42e4-98f2-f12330a7bb48\") " pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:46:59.682156 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.682123    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 15:46:59.682156 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.682145    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 15:46:59.682326 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.682160    2560 projected.go:196] Error preparing data for projected volume kube-api-access-5hxj6 for pod openshift-network-diagnostics/network-check-target-jxlsn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:46:59.682326 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:46:59.682220    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6 podName:72ca74e2-c3b6-42e4-98f2-f12330a7bb48 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:00.682200499 +0000 UTC m=+4.201567903 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hxj6" (UniqueName: "kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6") pod "network-check-target-jxlsn" (UID: "72ca74e2-c3b6-42e4-98f2-f12330a7bb48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:46:59.916237 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.916038    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 15:41:59 +0000 UTC" deadline="2027-12-24 08:02:41.609647526 +0000 UTC"
Jul 14 15:46:59.916237 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:46:59.916105    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12664h15m41.693546958s"
Jul 14 15:47:00.057647 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.056935    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h8lpb" event={"ID":"7b9dd8f7-527c-4dab-a52b-17c38f30fad1","Type":"ContainerStarted","Data":"c9f1d445dbe084b96fe43a7ed4817ef5a4c9e4e3cfa0f4f0e0bb1e6fca420f78"}
Jul 14 15:47:00.069916 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.069871    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g9l8z" event={"ID":"77a7601c-9b76-4736-94a3-0f9007c6c221","Type":"ContainerStarted","Data":"a8fe8d72965720adf17162df80cb3f0f0282a435b9b55c7534dcfbc1e1733bab"}
Jul 14 15:47:00.086896 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.086863    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-98fqw" event={"ID":"99983123-1bb8-4031-990a-bf3bfc8f5cda","Type":"ContainerStarted","Data":"72bc72d447c7910f7b645fbb691533aac3ab9c392436df4fa2b46a7950dc11c2"}
Jul 14 15:47:00.092952 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.092897    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sbkld" event={"ID":"c4921108-78c3-47e5-9965-2882382685de","Type":"ContainerStarted","Data":"783f71fd78e3b2daf85645b3ab44f371dd219bbf820134fa87be280f3cd3cbd9"}
Jul 14 15:47:00.113668 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.113550    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lx86m" event={"ID":"f5ac6baa-690d-477c-8090-199341500f68","Type":"ContainerStarted","Data":"849eac5ea5c68ae9abe02bc1f42da64bf86610b80c4becc6878631c2d3ffc376"}
Jul 14 15:47:00.146422 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.146384    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mtpnp" event={"ID":"0cee7827-8938-4376-8ba2-65a519b89996","Type":"ContainerStarted","Data":"395baced0ec0ceda38460dcad22ea818d09675157cc9b20bb3567432193c32bb"}
Jul 14 15:47:00.172376 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.172321    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" event={"ID":"7b65546031303e49503806457ed26462","Type":"ContainerStarted","Data":"d34007e209d501f231f03f0cdfec00fdc80e4536fdcf77f6d93036ff06d777d0"}
Jul 14 15:47:00.190946 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.190911    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s" event={"ID":"ff82c9a1-cbcf-485b-9fe0-6e24d41da713","Type":"ContainerStarted","Data":"66c8a0472c87b83d4442b1f0f6b22386296bb94907b3e36c0c3dd6d3e9a23cdb"}
Jul 14 15:47:00.202013 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.201979    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb" event={"ID":"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca","Type":"ContainerStarted","Data":"51b5d0c6366e446efdb4a91f5f542804c1e01ff6bb3d2c549a1ae366ae08483e"}
Jul 14 15:47:00.216425 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.216391    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lmv5z" event={"ID":"17d11734-7292-48a0-b53c-b3034fd7ae90","Type":"ContainerStarted","Data":"05d024df0010f5647203d8a38e3b740496f988fca1bca988f75ce02f1e4f2138"}
Jul 14 15:47:00.489283 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.489195    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:00.489463 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:00.489418    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:47:00.489554 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:00.489508    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs podName:9bbccbea-f177-4169-a89d-a09d860543b0 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:02.489466138 +0000 UTC m=+6.008833555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs") pod "network-metrics-daemon-w92wj" (UID: "9bbccbea-f177-4169-a89d-a09d860543b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:47:00.590853 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.590195    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:00.590853 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:00.590359    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 15:47:00.590853 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:00.590420    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret podName:0b801942-157e-4913-98a7-44d9440b3257 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:02.590400141 +0000 UTC m=+6.109767562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret") pod "global-pull-secret-syncer-8674z" (UID: "0b801942-157e-4913-98a7-44d9440b3257") : object "kube-system"/"original-pull-secret" not registered
Jul 14 15:47:00.593394 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.593368    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 15:47:00.691156 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.691122    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxj6\" (UniqueName: \"kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6\") pod \"network-check-target-jxlsn\" (UID: \"72ca74e2-c3b6-42e4-98f2-f12330a7bb48\") " pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:00.691348 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:00.691327    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 15:47:00.691437 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:00.691352    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 15:47:00.691437 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:00.691366    2560 projected.go:196] Error preparing data for projected volume kube-api-access-5hxj6 for pod openshift-network-diagnostics/network-check-target-jxlsn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:47:00.691437 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:00.691427    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6 podName:72ca74e2-c3b6-42e4-98f2-f12330a7bb48 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:02.691409096 +0000 UTC m=+6.210776709 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hxj6" (UniqueName: "kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6") pod "network-check-target-jxlsn" (UID: "72ca74e2-c3b6-42e4-98f2-f12330a7bb48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:47:00.916866 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.916761    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 15:41:59 +0000 UTC" deadline="2028-02-11 01:18:56.078845298 +0000 UTC"
Jul 14 15:47:00.916866 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:00.916808    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13833h31m55.162041329s"
Jul 14 15:47:01.028210 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:01.028181    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:01.028395 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:01.028328    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:01.028819 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:01.028763    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:01.028934 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:01.028906    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:01.029049 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:01.028994    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:01.029129 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:01.029094    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:02.516384 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:02.516231    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:02.516384 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:02.516391    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:47:02.516939 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:02.516450    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs podName:9bbccbea-f177-4169-a89d-a09d860543b0 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:06.51643582 +0000 UTC m=+10.035803221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs") pod "network-metrics-daemon-w92wj" (UID: "9bbccbea-f177-4169-a89d-a09d860543b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:47:02.617721 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:02.617112    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:02.617721 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:02.617277    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 15:47:02.617721 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:02.617343    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret podName:0b801942-157e-4913-98a7-44d9440b3257 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:06.617321505 +0000 UTC m=+10.136688907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret") pod "global-pull-secret-syncer-8674z" (UID: "0b801942-157e-4913-98a7-44d9440b3257") : object "kube-system"/"original-pull-secret" not registered
Jul 14 15:47:02.718892 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:02.718214    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxj6\" (UniqueName: \"kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6\") pod \"network-check-target-jxlsn\" (UID: \"72ca74e2-c3b6-42e4-98f2-f12330a7bb48\") " pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:02.718892 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:02.718418    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 15:47:02.718892 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:02.718438    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 15:47:02.718892 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:02.718451    2560 projected.go:196] Error preparing data for projected volume kube-api-access-5hxj6 for pod openshift-network-diagnostics/network-check-target-jxlsn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:47:02.718892 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:02.718531    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6 podName:72ca74e2-c3b6-42e4-98f2-f12330a7bb48 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:06.718510509 +0000 UTC m=+10.237877930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hxj6" (UniqueName: "kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6") pod "network-check-target-jxlsn" (UID: "72ca74e2-c3b6-42e4-98f2-f12330a7bb48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:47:03.026641 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:03.026610    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:03.026819 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:03.026770    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:03.026970 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:03.026947    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:03.027090 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:03.027066    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:03.027167 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:03.027132    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:03.027259 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:03.027241    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:05.029631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:05.028342    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:05.029631 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:05.029067    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:05.029631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:05.028738    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:05.029631 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:05.029168    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:05.029631 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:05.028695    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:05.029631 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:05.029243    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:06.552268 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:06.552159    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:06.552744 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:06.552385    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:47:06.552744 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:06.552493    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs podName:9bbccbea-f177-4169-a89d-a09d860543b0 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:14.552452304 +0000 UTC m=+18.071819716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs") pod "network-metrics-daemon-w92wj" (UID: "9bbccbea-f177-4169-a89d-a09d860543b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:47:06.653357 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:06.653308    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:06.653566 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:06.653476    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 15:47:06.653566 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:06.653558    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret podName:0b801942-157e-4913-98a7-44d9440b3257 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:14.653533836 +0000 UTC m=+18.172901260 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret") pod "global-pull-secret-syncer-8674z" (UID: "0b801942-157e-4913-98a7-44d9440b3257") : object "kube-system"/"original-pull-secret" not registered
Jul 14 15:47:06.754674 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:06.754070    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxj6\" (UniqueName: \"kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6\") pod \"network-check-target-jxlsn\" (UID: \"72ca74e2-c3b6-42e4-98f2-f12330a7bb48\") " pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:06.754674 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:06.754220    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 15:47:06.754674 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:06.754238    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 15:47:06.754674 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:06.754249    2560 projected.go:196] Error preparing data for projected volume kube-api-access-5hxj6 for pod openshift-network-diagnostics/network-check-target-jxlsn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:47:06.754674 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:06.754307    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6 podName:72ca74e2-c3b6-42e4-98f2-f12330a7bb48 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:14.754285481 +0000 UTC m=+18.273652932 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hxj6" (UniqueName: "kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6") pod "network-check-target-jxlsn" (UID: "72ca74e2-c3b6-42e4-98f2-f12330a7bb48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:47:07.030929 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:07.030898    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:07.031119 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:07.031071    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:07.031528 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:07.031493    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:07.031644 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:07.031624    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:07.031695 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:07.031683    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:07.031769 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:07.031752    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:09.029308 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:09.029270    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:09.029308 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:09.029308    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:09.029881 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:09.029320    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:09.029881 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:09.029415    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:09.029881 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:09.029483    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:09.029881 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:09.029601    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:11.026503 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:11.026462    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:11.026503 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:11.026470    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:11.026503 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:11.026468    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:11.027026 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:11.026610    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:11.027026 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:11.026638    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:11.027026 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:11.026713    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:13.026689 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:13.026655    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:13.027090 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:13.026651    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:13.027090 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:13.026789    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:13.027090 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:13.026817    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:13.027090 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:13.026907    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:13.027090 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:13.027000    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:14.620192 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:14.620156    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:14.620702 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:14.620332    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:47:14.620702 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:14.620409    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs podName:9bbccbea-f177-4169-a89d-a09d860543b0 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:30.620386823 +0000 UTC m=+34.139754244 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs") pod "network-metrics-daemon-w92wj" (UID: "9bbccbea-f177-4169-a89d-a09d860543b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:47:14.721002 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:14.720961    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:14.721182 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:14.721123    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 15:47:14.721251 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:14.721201    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret podName:0b801942-157e-4913-98a7-44d9440b3257 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:30.721181603 +0000 UTC m=+34.240549005 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret") pod "global-pull-secret-syncer-8674z" (UID: "0b801942-157e-4913-98a7-44d9440b3257") : object "kube-system"/"original-pull-secret" not registered
Jul 14 15:47:14.821466 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:14.821413    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxj6\" (UniqueName: \"kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6\") pod \"network-check-target-jxlsn\" (UID: \"72ca74e2-c3b6-42e4-98f2-f12330a7bb48\") " pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:14.821625 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:14.821580    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 15:47:14.821625 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:14.821608    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 15:47:14.821625 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:14.821622    2560 projected.go:196] Error preparing data for projected volume kube-api-access-5hxj6 for pod openshift-network-diagnostics/network-check-target-jxlsn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:47:14.821748 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:14.821676    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6 podName:72ca74e2-c3b6-42e4-98f2-f12330a7bb48 nodeName:}" failed. No retries permitted until 2026-07-14 15:47:30.821661069 +0000 UTC m=+34.341028468 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hxj6" (UniqueName: "kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6") pod "network-check-target-jxlsn" (UID: "72ca74e2-c3b6-42e4-98f2-f12330a7bb48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:47:15.025954 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:15.025884    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:15.026113 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:15.025887    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:15.026113 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:15.026030    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:15.026113 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:15.026043    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:15.026264 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:15.026149    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:15.026264 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:15.026235    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:17.027662 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:17.027504    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:17.027662 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:17.027532    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:17.027662 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:17.027622    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:17.028233 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:17.027682    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:17.028233 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:17.027746    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:17.028233 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:17.027827    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:19.026693 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:19.026648    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:19.027199 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:19.026650    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:19.027199 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:19.026790    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:19.027199 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:19.026801    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:19.027199 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:19.026911    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:19.027199 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:19.026989    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:21.027315 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.026741    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:21.027315 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:21.027065    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:21.027315 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.027106    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:21.027315 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:21.027241    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:21.027315 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.026959    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:21.027889 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:21.027333    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:21.275570 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.275546    2560 generic.go:356] "Generic (PLEG): container finished" podID="77a7601c-9b76-4736-94a3-0f9007c6c221" containerID="b8c9535dfa856d7651ada24aee4b9b44e23e3691f001b2e46574a708e566af45" exitCode=0
Jul 14 15:47:21.275674 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.275615    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g9l8z" event={"ID":"77a7601c-9b76-4736-94a3-0f9007c6c221","Type":"ContainerDied","Data":"b8c9535dfa856d7651ada24aee4b9b44e23e3691f001b2e46574a708e566af45"}
Jul 14 15:47:21.277082 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.277059    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sbkld" event={"ID":"c4921108-78c3-47e5-9965-2882382685de","Type":"ContainerStarted","Data":"aa381edd5c7a1d9223d23fcd8fff44d772cdcf29e8055c8591a8df0df38c519f"}
Jul 14 15:47:21.278438 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.278413    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lx86m" event={"ID":"f5ac6baa-690d-477c-8090-199341500f68","Type":"ContainerStarted","Data":"d9e05ceba2b4fcdc2b410ae751ba0f0fdaf8b9ae24a90072f4428ca536acbd83"}
Jul 14 15:47:21.279963 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.279911    2560 generic.go:356] "Generic (PLEG): container finished" podID="0009cb587a3b9c89871f7a780a349638" containerID="230aaeb3d89385ecb2996728a270a5859e80c794ea4860899bbb57b6f3ee0f8c" exitCode=0
Jul 14 15:47:21.280023 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.279998    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" event={"ID":"0009cb587a3b9c89871f7a780a349638","Type":"ContainerDied","Data":"230aaeb3d89385ecb2996728a270a5859e80c794ea4860899bbb57b6f3ee0f8c"}
Jul 14 15:47:21.281302 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.281280    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mtpnp" event={"ID":"0cee7827-8938-4376-8ba2-65a519b89996","Type":"ContainerStarted","Data":"bbc9d69d7986188394e69e5404e970cd6c31fe5baf12ed3d0dd0c68eb66c67b4"}
Jul 14 15:47:21.282686 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.282665    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" event={"ID":"7b65546031303e49503806457ed26462","Type":"ContainerStarted","Data":"581b77dce368acd9b3e2537324923ba9409e40fe3fe00ba7b91bb57d2ef7557d"}
Jul 14 15:47:21.284945 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.284930    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 15:47:21.285284 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.285255    2560 generic.go:356] "Generic (PLEG): container finished" podID="ff82c9a1-cbcf-485b-9fe0-6e24d41da713" containerID="bfeaf69f6ae2632b6eab7ce7f20f3b4189ba2fc4682a023899edcf7a731d562d" exitCode=1
Jul 14 15:47:21.285374 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.285318    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s" event={"ID":"ff82c9a1-cbcf-485b-9fe0-6e24d41da713","Type":"ContainerStarted","Data":"9082faed7287be1d32b2b20a1ce44a3d56656ddc46bba2d491fd7dd7ea4666bb"}
Jul 14 15:47:21.285431 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.285409    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s" event={"ID":"ff82c9a1-cbcf-485b-9fe0-6e24d41da713","Type":"ContainerStarted","Data":"472ac7eb94e0cf134675b8469c442e7c7fa639a360217839da00122a596954dc"}
Jul 14 15:47:21.285578 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.285480    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s" event={"ID":"ff82c9a1-cbcf-485b-9fe0-6e24d41da713","Type":"ContainerStarted","Data":"1d6b972066120aca6c214fbd1aaa6e93ce8ba5acbf72fdc183e502a20d163516"}
Jul 14 15:47:21.285578 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.285536    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s" event={"ID":"ff82c9a1-cbcf-485b-9fe0-6e24d41da713","Type":"ContainerDied","Data":"bfeaf69f6ae2632b6eab7ce7f20f3b4189ba2fc4682a023899edcf7a731d562d"}
Jul 14 15:47:21.285675 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.285578    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s" event={"ID":"ff82c9a1-cbcf-485b-9fe0-6e24d41da713","Type":"ContainerStarted","Data":"4b5793a589d8d680b73ba4d73cc303ec1be4ba1430d216a3326235c67fe2bbda"}
Jul 14 15:47:21.286874 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.286805    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb" event={"ID":"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca","Type":"ContainerStarted","Data":"0a418858ec9b6dc0b87dca426a7edb70740cece64c0c85897cab450bdd1be310"}
Jul 14 15:47:21.288154 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.288138    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lmv5z" event={"ID":"17d11734-7292-48a0-b53c-b3034fd7ae90","Type":"ContainerStarted","Data":"a8cc685a01eee9a2aa3b78de12ee1d8afc30e4b00583d4a2ff52bb09f6c33a2b"}
Jul 14 15:47:21.289706 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.289688    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h8lpb" event={"ID":"7b9dd8f7-527c-4dab-a52b-17c38f30fad1","Type":"ContainerStarted","Data":"14173ba5431a0d8cc4c0d31419fb3bde7581b1d6d3bca4064b3674ef9e25529a"}
Jul 14 15:47:21.375787 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.375699    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h8lpb" podStartSLOduration=2.940684865 podStartE2EDuration="24.375678277s" totalImagesPullingTime="21.434993412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:21.373851356 +0000 UTC m=+24.893218776" watchObservedRunningTime="2026-07-14 15:47:21.375678277 +0000 UTC m=+24.895045697"
Jul 14 15:47:21.376024 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.375981    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mtpnp" podStartSLOduration=1.999135979 podStartE2EDuration="23.375971779s" totalImagesPullingTime="21.3768358s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:21.339925316 +0000 UTC m=+24.859292739" watchObservedRunningTime="2026-07-14 15:47:21.375971779 +0000 UTC m=+24.895339200"
Jul 14 15:47:21.420105 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.420049    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lmv5z" podStartSLOduration=1.989481971 podStartE2EDuration="23.420028974s" totalImagesPullingTime="21.430547003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:21.418596542 +0000 UTC m=+24.937963966" watchObservedRunningTime="2026-07-14 15:47:21.420028974 +0000 UTC m=+24.939396397"
Jul 14 15:47:21.479744 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.479690    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lx86m" podStartSLOduration=7.593721172 podStartE2EDuration="24.47967361s" totalImagesPullingTime="16.885952438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:21.479114621 +0000 UTC m=+24.998482042" watchObservedRunningTime="2026-07-14 15:47:21.47967361 +0000 UTC m=+24.999041030"
Jul 14 15:47:21.479907 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.479778    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-20.ec2.internal" podStartSLOduration=23.479770758 podStartE2EDuration="23.479770758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:21.452611572 +0000 UTC m=+24.971978993" watchObservedRunningTime="2026-07-14 15:47:21.479770758 +0000 UTC m=+24.999138181"
Jul 14 15:47:21.513794 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:21.513747    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sbkld" podStartSLOduration=3.108510597 podStartE2EDuration="24.513729591s" totalImagesPullingTime="21.405218994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:21.513282042 +0000 UTC m=+25.032649465" watchObservedRunningTime="2026-07-14 15:47:21.513729591 +0000 UTC m=+25.033097009"
Jul 14 15:47:22.245181 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:22.245152    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 14 15:47:22.295221 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:22.295147    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 15:47:22.295635 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:22.295610    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s" event={"ID":"ff82c9a1-cbcf-485b-9fe0-6e24d41da713","Type":"ContainerStarted","Data":"f5931d774914631e48c4f210dbf3d277b468ae9dde62ee6e7cab0992b6e56112"}
Jul 14 15:47:22.297674 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:22.297643    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb" event={"ID":"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca","Type":"ContainerStarted","Data":"4e657c668d97429dbf354cf5457c44575ad5cb204b2ef807e5227f272dbaaad3"}
Jul 14 15:47:22.299287 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:22.299250    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-98fqw" event={"ID":"99983123-1bb8-4031-990a-bf3bfc8f5cda","Type":"ContainerStarted","Data":"85514fba37516c071914f8efae5ee2db961d2ae6d65e43e8d262adf3917fbfda"}
Jul 14 15:47:22.328018 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:22.327961    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-98fqw" podStartSLOduration=3.927785259 podStartE2EDuration="25.327942078s" totalImagesPullingTime="21.400156819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:22.327741758 +0000 UTC m=+25.847109204" watchObservedRunningTime="2026-07-14 15:47:22.327942078 +0000 UTC m=+25.847309501"
Jul 14 15:47:22.929800 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:22.929693    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T15:47:22.245178571Z","UUID":"dbbeb208-c9ca-4aae-956f-7f2a4a3afd9b","Handler":null,"Name":"","Endpoint":""}
Jul 14 15:47:22.931376 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:22.931355    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 14 15:47:22.931512 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:22.931399    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 14 15:47:23.026779 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:23.026747    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:23.026955 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:23.026788    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:23.026955 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:23.026749    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:23.026955 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:23.026870    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:23.027160 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:23.026994    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:23.027160 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:23.027056    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:23.118093 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:23.118056    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:47:23.118679 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:23.118657    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:47:23.169748 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:23.169723    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mtpnp_0cee7827-8938-4376-8ba2-65a519b89996/dns-node-resolver/0.log"
Jul 14 15:47:23.303206 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:23.303130    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" event={"ID":"0009cb587a3b9c89871f7a780a349638","Type":"ContainerStarted","Data":"704f386f92c407a64ad5b5e9cd1cc944fbe24ac32e53ad3c2e8fdba2ee8a5ff2"}
Jul 14 15:47:23.303657 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:23.303582    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:47:23.303932 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:23.303914    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lx86m"
Jul 14 15:47:23.321921 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:23.321880    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-20.ec2.internal" podStartSLOduration=25.32186565 podStartE2EDuration="25.32186565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:23.321750709 +0000 UTC m=+26.841118131" watchObservedRunningTime="2026-07-14 15:47:23.32186565 +0000 UTC m=+26.841233072"
Jul 14 15:47:23.949178 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:23.948958    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sbkld_c4921108-78c3-47e5-9965-2882382685de/node-ca/0.log"
Jul 14 15:47:24.309662 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:24.309597    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 15:47:24.310093 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:24.310036    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s" event={"ID":"ff82c9a1-cbcf-485b-9fe0-6e24d41da713","Type":"ContainerStarted","Data":"fcf9d2d85a5022c045a1bb605a0ec07f055962137a90480cb7c16564b1c6431a"}
Jul 14 15:47:25.026128 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:25.026055    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:25.026291 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:25.026055    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:25.026291 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:25.026155    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:25.026291 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:25.026222    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:25.026291 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:25.026060    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:25.026568 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:25.026299    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:26.316895 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:26.316707    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 15:47:26.317571 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:26.317217    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s" event={"ID":"ff82c9a1-cbcf-485b-9fe0-6e24d41da713","Type":"ContainerStarted","Data":"5185168cb2d97d8d62b5f5a72c7045c041fde9b13a2873625c48fcb861b52059"}
Jul 14 15:47:26.317571 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:26.317544    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:47:26.317667 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:26.317633    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:47:26.317713 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:26.317693    2560 scope.go:117] "RemoveContainer" containerID="bfeaf69f6ae2632b6eab7ce7f20f3b4189ba2fc4682a023899edcf7a731d562d"
Jul 14 15:47:26.323528 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:26.323487    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb" event={"ID":"631b26cf-cb79-47a0-9e44-dcf2f5a1d0ca","Type":"ContainerStarted","Data":"044aaec27d269e2b3db7d2e9e2acdecd4878c0ea3ce0ecbcac05b8b1b72ded8e"}
Jul 14 15:47:26.325174 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:26.325150    2560 generic.go:356] "Generic (PLEG): container finished" podID="77a7601c-9b76-4736-94a3-0f9007c6c221" containerID="5e04bf791f355a9bfb16ce4f35297a38919f004b99d5e69c5830700a259e7372" exitCode=0
Jul 14 15:47:26.325253 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:26.325189    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g9l8z" event={"ID":"77a7601c-9b76-4736-94a3-0f9007c6c221","Type":"ContainerDied","Data":"5e04bf791f355a9bfb16ce4f35297a38919f004b99d5e69c5830700a259e7372"}
Jul 14 15:47:26.335628 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:26.335609    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:47:26.375589 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:26.375551    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qh2fb" podStartSLOduration=2.340133601 podStartE2EDuration="28.37553698s" totalImagesPullingTime="26.035403379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:26.375319112 +0000 UTC m=+29.894686535" watchObservedRunningTime="2026-07-14 15:47:26.37553698 +0000 UTC m=+29.894904401"
Jul 14 15:47:27.028662 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:27.028622    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:27.028852 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:27.028788    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:27.029270 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:27.029249    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:27.029388 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:27.029367    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:27.029465 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:27.029439    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:27.029570 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:27.029553    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:27.330337 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:27.330311    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 15:47:27.330759 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:27.330716    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s" event={"ID":"ff82c9a1-cbcf-485b-9fe0-6e24d41da713","Type":"ContainerStarted","Data":"99353814c9ec0c2fd6ed0a56cf15123a49dde0964dc6ebaf25149ec1aab58474"}
Jul 14 15:47:27.330981 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:27.330962    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:47:27.346975 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:27.346953    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:47:27.375805 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:27.375757    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s" podStartSLOduration=7.8999116019999995 podStartE2EDuration="29.375741803s" totalImagesPullingTime="21.475830201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:27.375166248 +0000 UTC m=+30.894533669" watchObservedRunningTime="2026-07-14 15:47:27.375741803 +0000 UTC m=+30.895109224"
Jul 14 15:47:28.335237 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:28.335050    2560 generic.go:356] "Generic (PLEG): container finished" podID="77a7601c-9b76-4736-94a3-0f9007c6c221" containerID="542f15a33e2ccdfc157bed349684ebcd038aba0a0d91fc4b413728a02c0ca5b9" exitCode=0
Jul 14 15:47:28.335790 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:28.335137    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g9l8z" event={"ID":"77a7601c-9b76-4736-94a3-0f9007c6c221","Type":"ContainerDied","Data":"542f15a33e2ccdfc157bed349684ebcd038aba0a0d91fc4b413728a02c0ca5b9"}
Jul 14 15:47:29.028760 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:29.028727    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:29.028914 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:29.028812    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:29.028914 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:29.028821    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:29.028914 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:29.028837    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:29.029014 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:29.028917    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:29.029014 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:29.028979    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:30.341726 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:30.341687    2560 generic.go:356] "Generic (PLEG): container finished" podID="77a7601c-9b76-4736-94a3-0f9007c6c221" containerID="24159e2dd0fc947644324acb2918dd65a315103b7efe7ab64c2a49db9566e583" exitCode=0
Jul 14 15:47:30.342171 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:30.341747    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g9l8z" event={"ID":"77a7601c-9b76-4736-94a3-0f9007c6c221","Type":"ContainerDied","Data":"24159e2dd0fc947644324acb2918dd65a315103b7efe7ab64c2a49db9566e583"}
Jul 14 15:47:30.640183 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:30.640096    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:30.640321 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:30.640200    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:47:30.640321 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:30.640258    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs podName:9bbccbea-f177-4169-a89d-a09d860543b0 nodeName:}" failed. No retries permitted until 2026-07-14 15:48:02.640245098 +0000 UTC m=+66.159612497 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs") pod "network-metrics-daemon-w92wj" (UID: "9bbccbea-f177-4169-a89d-a09d860543b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 15:47:30.741029 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:30.740999    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:30.741171 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:30.741112    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 15:47:30.741171 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:30.741166    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret podName:0b801942-157e-4913-98a7-44d9440b3257 nodeName:}" failed. No retries permitted until 2026-07-14 15:48:02.741151593 +0000 UTC m=+66.260518995 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret") pod "global-pull-secret-syncer-8674z" (UID: "0b801942-157e-4913-98a7-44d9440b3257") : object "kube-system"/"original-pull-secret" not registered
Jul 14 15:47:30.841364 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:30.841327    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxj6\" (UniqueName: \"kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6\") pod \"network-check-target-jxlsn\" (UID: \"72ca74e2-c3b6-42e4-98f2-f12330a7bb48\") " pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:30.841557 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:30.841501    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 15:47:30.841557 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:30.841521    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 15:47:30.841557 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:30.841531    2560 projected.go:196] Error preparing data for projected volume kube-api-access-5hxj6 for pod openshift-network-diagnostics/network-check-target-jxlsn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:47:30.841677 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:30.841581    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6 podName:72ca74e2-c3b6-42e4-98f2-f12330a7bb48 nodeName:}" failed. No retries permitted until 2026-07-14 15:48:02.841568665 +0000 UTC m=+66.360936063 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5hxj6" (UniqueName: "kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6") pod "network-check-target-jxlsn" (UID: "72ca74e2-c3b6-42e4-98f2-f12330a7bb48") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 15:47:31.026404 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:31.026322    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:31.026404 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:31.026356    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:31.026404 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:31.026375    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:31.026621 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:31.026465    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:31.026621 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:31.026593    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:31.026694 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:31.026655    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:33.026830 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:33.026798    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:33.027301 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:33.026941    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:33.027301 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:33.026809    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:33.027301 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:33.027033    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:33.027301 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:33.026806    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:33.027301 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:33.027104    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:35.026041 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:35.026009    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:35.026504 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:35.026019    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:35.026504 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:35.026131    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:35.026504 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:35.026129    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:35.026504 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:35.026203    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:35.026504 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:35.026276    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:36.361487 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:36.361431    2560 generic.go:356] "Generic (PLEG): container finished" podID="77a7601c-9b76-4736-94a3-0f9007c6c221" containerID="b5758649ef83767ddc0971f253050dc1a62c4f2fb415124090db680dca2b4f5d" exitCode=0
Jul 14 15:47:36.361852 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:36.361506    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g9l8z" event={"ID":"77a7601c-9b76-4736-94a3-0f9007c6c221","Type":"ContainerDied","Data":"b5758649ef83767ddc0971f253050dc1a62c4f2fb415124090db680dca2b4f5d"}
Jul 14 15:47:37.026989 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:37.026957    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:37.027147 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:37.027033    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:37.027147 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:37.027050    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:37.027147 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:37.027116    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:37.027258 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:37.027175    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:37.027258 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:37.027173    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:37.366229 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:37.366198    2560 generic.go:356] "Generic (PLEG): container finished" podID="77a7601c-9b76-4736-94a3-0f9007c6c221" containerID="f645a14e8e32608a4bbdfdbf8470da0fe2dab112328e7220941cb4dc9424d359" exitCode=0
Jul 14 15:47:37.366668 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:37.366251    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g9l8z" event={"ID":"77a7601c-9b76-4736-94a3-0f9007c6c221","Type":"ContainerDied","Data":"f645a14e8e32608a4bbdfdbf8470da0fe2dab112328e7220941cb4dc9424d359"}
Jul 14 15:47:38.371975 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:38.371940    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g9l8z" event={"ID":"77a7601c-9b76-4736-94a3-0f9007c6c221","Type":"ContainerStarted","Data":"77ecb4d959db826a9038b80b91da1c0bb5205e11415ccb8801ffe32d6f71cbc0"}
Jul 14 15:47:38.399896 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:38.399849    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-g9l8z" podStartSLOduration=8.203966483 podStartE2EDuration="41.399836266s" totalImagesPullingTime="33.195869783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:47:38.39964547 +0000 UTC m=+41.919012891" watchObservedRunningTime="2026-07-14 15:47:38.399836266 +0000 UTC m=+41.919203687"
Jul 14 15:47:39.026113 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:39.026083    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:39.026273 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:39.026180    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:39.026273 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:39.026199    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:39.026418 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:39.026179    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:39.026418 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:39.026300    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:39.026556 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:39.026409    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:41.025719 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:41.025687    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:41.025719 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:41.025702    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:41.026148 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:41.025697    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:41.026148 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:41.025793    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:41.026148 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:41.025882    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:41.026148 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:41.025967    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:43.026401 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:43.026358    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:43.026401 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:43.026376    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:43.026950 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:43.026512    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:43.026950 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:43.026584    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:43.026950 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:43.026650    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:43.026950 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:43.026768    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:45.026266 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:45.026222    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:45.026732 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:45.026223    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:45.026732 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:45.026335    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:45.026732 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:45.026353    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:45.026732 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:45.026426    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:45.026732 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:45.026535    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:47.027209 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:47.027165    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:47.027682 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:47.027282    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:47.027682 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:47.027292    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:47.027682 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:47.027318    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:47.027682 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:47.027368    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:47.027682 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:47.027410    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:48.285136 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:48.284957    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jxlsn"]
Jul 14 15:47:48.285606 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:48.285212    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:48.285606 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:48.285307    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:48.288844 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:48.288816    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8674z"]
Jul 14 15:47:48.288980 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:48.288907    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:48.289038 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:48.288999    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:48.289586 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:48.289561    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w92wj"]
Jul 14 15:47:48.289681 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:48.289645    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:48.289765 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:48.289741    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:50.025774 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:50.025742    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:50.026303 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:50.025842    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:50.026303 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:50.025843    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:50.026303 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:50.025849    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:50.026303 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:50.025950    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:50.026303 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:50.026004    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:52.026422 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.026385    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:52.026422 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.026409    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:52.026835 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.026431    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:52.026835 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:52.026523    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w92wj" podUID="9bbccbea-f177-4169-a89d-a09d860543b0"
Jul 14 15:47:52.026835 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:52.026559    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8674z" podUID="0b801942-157e-4913-98a7-44d9440b3257"
Jul 14 15:47:52.026835 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:47:52.026613    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jxlsn" podUID="72ca74e2-c3b6-42e4-98f2-f12330a7bb48"
Jul 14 15:47:52.267089 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.267063    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-20.ec2.internal" event="NodeReady"
Jul 14 15:47:52.267223 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.267179    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 14 15:47:52.338823 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.338800    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nwpfv"]
Jul 14 15:47:52.353867 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.353846    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8mzm5"]
Jul 14 15:47:52.353949 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.353851    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nwpfv"
Jul 14 15:47:52.357275 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.357256    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 14 15:47:52.359783 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.359756    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 14 15:47:52.359904 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.359861    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9twdb\""
Jul 14 15:47:52.360292 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.360273    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 14 15:47:52.364070 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.364054    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.367344 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.367330    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-km4cd\""
Jul 14 15:47:52.367827 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.367811    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 14 15:47:52.367918 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.367814    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 14 15:47:52.372376 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.372353    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8mzm5"]
Jul 14 15:47:52.372927 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.372911    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nwpfv"]
Jul 14 15:47:52.399807 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.399787    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec667542-ec92-4054-a5d5-849cfe374814-tmp-dir\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.399943 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.399816    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbj4h\" (UniqueName: \"kubernetes.io/projected/e6b29cdc-5f48-48ad-b97a-42a968c2ce7d-kube-api-access-gbj4h\") pod \"ingress-canary-nwpfv\" (UID: \"e6b29cdc-5f48-48ad-b97a-42a968c2ce7d\") " pod="openshift-ingress-canary/ingress-canary-nwpfv"
Jul 14 15:47:52.399943 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.399847    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6b29cdc-5f48-48ad-b97a-42a968c2ce7d-cert\") pod \"ingress-canary-nwpfv\" (UID: \"e6b29cdc-5f48-48ad-b97a-42a968c2ce7d\") " pod="openshift-ingress-canary/ingress-canary-nwpfv"
Jul 14 15:47:52.399943 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.399914    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec667542-ec92-4054-a5d5-849cfe374814-metrics-tls\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.400055 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.399964    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsmsl\" (UniqueName: \"kubernetes.io/projected/ec667542-ec92-4054-a5d5-849cfe374814-kube-api-access-gsmsl\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.400055 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.400040    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec667542-ec92-4054-a5d5-849cfe374814-config-volume\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.444743 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.444715    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zkc2d"]
Jul 14 15:47:52.468223 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.468198    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zkc2d"]
Jul 14 15:47:52.468353 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.468310    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.473776 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.473739    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 14 15:47:52.474420 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.474396    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lf679\""
Jul 14 15:47:52.474559 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.474541    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 14 15:47:52.475218 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.475188    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 14 15:47:52.478315 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.478294    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 14 15:47:52.500482 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.500439    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6b29cdc-5f48-48ad-b97a-42a968c2ce7d-cert\") pod \"ingress-canary-nwpfv\" (UID: \"e6b29cdc-5f48-48ad-b97a-42a968c2ce7d\") " pod="openshift-ingress-canary/ingress-canary-nwpfv"
Jul 14 15:47:52.500582 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.500497    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec667542-ec92-4054-a5d5-849cfe374814-metrics-tls\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.500582 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.500524    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsmsl\" (UniqueName: \"kubernetes.io/projected/ec667542-ec92-4054-a5d5-849cfe374814-kube-api-access-gsmsl\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.500688 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.500594    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec667542-ec92-4054-a5d5-849cfe374814-config-volume\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.500688 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.500625    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec667542-ec92-4054-a5d5-849cfe374814-tmp-dir\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.500688 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.500652    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbj4h\" (UniqueName: \"kubernetes.io/projected/e6b29cdc-5f48-48ad-b97a-42a968c2ce7d-kube-api-access-gbj4h\") pod \"ingress-canary-nwpfv\" (UID: \"e6b29cdc-5f48-48ad-b97a-42a968c2ce7d\") " pod="openshift-ingress-canary/ingress-canary-nwpfv"
Jul 14 15:47:52.501158 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.501133    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec667542-ec92-4054-a5d5-849cfe374814-tmp-dir\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.501387 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.501363    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec667542-ec92-4054-a5d5-849cfe374814-config-volume\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.504779 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.504755    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec667542-ec92-4054-a5d5-849cfe374814-metrics-tls\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.504868 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.504799    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6b29cdc-5f48-48ad-b97a-42a968c2ce7d-cert\") pod \"ingress-canary-nwpfv\" (UID: \"e6b29cdc-5f48-48ad-b97a-42a968c2ce7d\") " pod="openshift-ingress-canary/ingress-canary-nwpfv"
Jul 14 15:47:52.515847 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.515821    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbj4h\" (UniqueName: \"kubernetes.io/projected/e6b29cdc-5f48-48ad-b97a-42a968c2ce7d-kube-api-access-gbj4h\") pod \"ingress-canary-nwpfv\" (UID: \"e6b29cdc-5f48-48ad-b97a-42a968c2ce7d\") " pod="openshift-ingress-canary/ingress-canary-nwpfv"
Jul 14 15:47:52.516262 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.516242    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsmsl\" (UniqueName: \"kubernetes.io/projected/ec667542-ec92-4054-a5d5-849cfe374814-kube-api-access-gsmsl\") pod \"dns-default-8mzm5\" (UID: \"ec667542-ec92-4054-a5d5-849cfe374814\") " pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.601590 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.601508    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8c6cfcb-efe3-4838-b2d9-614bd266711a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.601590 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.601550    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8c6cfcb-efe3-4838-b2d9-614bd266711a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.601590 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.601570    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8c6cfcb-efe3-4838-b2d9-614bd266711a-data-volume\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.601790 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.601614    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e8c6cfcb-efe3-4838-b2d9-614bd266711a-crio-socket\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.601790 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.601653    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2w5h\" (UniqueName: \"kubernetes.io/projected/e8c6cfcb-efe3-4838-b2d9-614bd266711a-kube-api-access-r2w5h\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.663503 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.663479    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nwpfv"
Jul 14 15:47:52.672519 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.672490    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:47:52.702632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.702601    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e8c6cfcb-efe3-4838-b2d9-614bd266711a-crio-socket\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.702781 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.702642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2w5h\" (UniqueName: \"kubernetes.io/projected/e8c6cfcb-efe3-4838-b2d9-614bd266711a-kube-api-access-r2w5h\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.702781 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.702715    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8c6cfcb-efe3-4838-b2d9-614bd266711a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.702781 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.702749    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8c6cfcb-efe3-4838-b2d9-614bd266711a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.702926 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.702845    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8c6cfcb-efe3-4838-b2d9-614bd266711a-data-volume\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.703252 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.703189    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8c6cfcb-efe3-4838-b2d9-614bd266711a-data-volume\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.703363 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.703258    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8c6cfcb-efe3-4838-b2d9-614bd266711a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.703363 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.703296    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e8c6cfcb-efe3-4838-b2d9-614bd266711a-crio-socket\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.705626 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.705603    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8c6cfcb-efe3-4838-b2d9-614bd266711a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.722861 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.722833    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2w5h\" (UniqueName: \"kubernetes.io/projected/e8c6cfcb-efe3-4838-b2d9-614bd266711a-kube-api-access-r2w5h\") pod \"insights-runtime-extractor-zkc2d\" (UID: \"e8c6cfcb-efe3-4838-b2d9-614bd266711a\") " pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.778022 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.777750    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zkc2d"
Jul 14 15:47:52.824432 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.824376    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nwpfv"]
Jul 14 15:47:52.825640 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.825617    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8mzm5"]
Jul 14 15:47:52.829753 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:47:52.829724    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec667542_ec92_4054_a5d5_849cfe374814.slice/crio-a22c11b268bb0208bb1edaff0418cbfeee08d58b2447662ff0fb6285298a49b9 WatchSource:0}: Error finding container a22c11b268bb0208bb1edaff0418cbfeee08d58b2447662ff0fb6285298a49b9: Status 404 returned error can't find the container with id a22c11b268bb0208bb1edaff0418cbfeee08d58b2447662ff0fb6285298a49b9
Jul 14 15:47:52.830134 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:47:52.830110    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b29cdc_5f48_48ad_b97a_42a968c2ce7d.slice/crio-41f4ae6a1eea1d55b902c165ae5b6ddd8905f1230da394926b1d0ceb70085d1d WatchSource:0}: Error finding container 41f4ae6a1eea1d55b902c165ae5b6ddd8905f1230da394926b1d0ceb70085d1d: Status 404 returned error can't find the container with id 41f4ae6a1eea1d55b902c165ae5b6ddd8905f1230da394926b1d0ceb70085d1d
Jul 14 15:47:52.839246 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.839227    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fb758b489-l2g29"]
Jul 14 15:47:52.872466 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.870215    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fb758b489-l2g29"]
Jul 14 15:47:52.872466 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.870334    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:52.876462 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.874351    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qvsqv\""
Jul 14 15:47:52.876462 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.874705    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 14 15:47:52.876462 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.874885    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 14 15:47:52.876462 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.875635    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 14 15:47:52.876462 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.875843    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 14 15:47:52.876462 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.876190    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 14 15:47:52.876797 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.876569    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 14 15:47:52.876842 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.876823    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 14 15:47:52.959936 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:52.959908    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zkc2d"]
Jul 14 15:47:52.964029 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:47:52.964002    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c6cfcb_efe3_4838_b2d9_614bd266711a.slice/crio-026ae33b64b0e543be6eeb127e68a60afaaf9b2db56e1abd71dc48708fcab71e WatchSource:0}: Error finding container 026ae33b64b0e543be6eeb127e68a60afaaf9b2db56e1abd71dc48708fcab71e: Status 404 returned error can't find the container with id 026ae33b64b0e543be6eeb127e68a60afaaf9b2db56e1abd71dc48708fcab71e
Jul 14 15:47:53.004745 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.004716    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-oauth-serving-cert\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.004745 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.004758    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-serving-cert\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.004921 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.004788    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-oauth-config\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.004921 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.004840    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrc9f\" (UniqueName: \"kubernetes.io/projected/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-kube-api-access-vrc9f\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.004921 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.004901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-config\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.004921 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.004917    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-service-ca\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.105875 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.105835    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-config\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.105875 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.105866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-service-ca\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.106576 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.105915    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-oauth-serving-cert\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.106576 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.105951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-serving-cert\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.106576 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.105980    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-oauth-config\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.106576 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.106003    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrc9f\" (UniqueName: \"kubernetes.io/projected/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-kube-api-access-vrc9f\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.106863 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.106841    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-config\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.106960 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.106937    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-service-ca\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.107016 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.106937    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-oauth-serving-cert\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.108951 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.108933    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-serving-cert\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.109029 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.108958    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-oauth-config\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.115161 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.115113    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrc9f\" (UniqueName: \"kubernetes.io/projected/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-kube-api-access-vrc9f\") pod \"console-5fb758b489-l2g29\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") " pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.184281 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.184253    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:47:53.326942 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.326912    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fb758b489-l2g29"]
Jul 14 15:47:53.330614 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:47:53.330584    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode74e6c41_9677_4bbf_a1b5_92ffe45df5ec.slice/crio-4caa4cf1904a11db1606c9d0c6cd2a139eeb31cd3317f0681bcb9958a68d564c WatchSource:0}: Error finding container 4caa4cf1904a11db1606c9d0c6cd2a139eeb31cd3317f0681bcb9958a68d564c: Status 404 returned error can't find the container with id 4caa4cf1904a11db1606c9d0c6cd2a139eeb31cd3317f0681bcb9958a68d564c
Jul 14 15:47:53.405710 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.405644    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nwpfv" event={"ID":"e6b29cdc-5f48-48ad-b97a-42a968c2ce7d","Type":"ContainerStarted","Data":"41f4ae6a1eea1d55b902c165ae5b6ddd8905f1230da394926b1d0ceb70085d1d"}
Jul 14 15:47:53.407058 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.407027    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb758b489-l2g29" event={"ID":"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec","Type":"ContainerStarted","Data":"4caa4cf1904a11db1606c9d0c6cd2a139eeb31cd3317f0681bcb9958a68d564c"}
Jul 14 15:47:53.408407 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.408381    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zkc2d" event={"ID":"e8c6cfcb-efe3-4838-b2d9-614bd266711a","Type":"ContainerStarted","Data":"026ae33b64b0e543be6eeb127e68a60afaaf9b2db56e1abd71dc48708fcab71e"}
Jul 14 15:47:53.409697 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:53.409645    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8mzm5" event={"ID":"ec667542-ec92-4054-a5d5-849cfe374814","Type":"ContainerStarted","Data":"a22c11b268bb0208bb1edaff0418cbfeee08d58b2447662ff0fb6285298a49b9"}
Jul 14 15:47:54.027162 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:54.026123    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:47:54.027162 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:54.026471    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:47:54.027162 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:54.026792    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:47:54.032107 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:54.031370    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 15:47:54.032107 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:54.031496    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 15:47:54.032107 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:54.031599    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 15:47:54.032107 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:54.031811    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zfvj8\""
Jul 14 15:47:54.032107 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:54.031829    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 15:47:54.032107 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:54.032033    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gt8h7\""
Jul 14 15:47:55.707661 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.707620    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-fstjl"]
Jul 14 15:47:55.726661 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.726628    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-fstjl"]
Jul 14 15:47:55.726813 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.726727    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.731401 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.731376    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 14 15:47:55.731628 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.731613    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 14 15:47:55.732052 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.732029    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8rh2j\""
Jul 14 15:47:55.732615 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.732598    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 14 15:47:55.732720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.732664    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 14 15:47:55.732720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.732705    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 14 15:47:55.831126 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.831092    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-499sg\" (UniqueName: \"kubernetes.io/projected/43282f36-5709-462b-9d94-117b37c45a74-kube-api-access-499sg\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.831357 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.831211    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/43282f36-5709-462b-9d94-117b37c45a74-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.831357 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.831239    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43282f36-5709-462b-9d94-117b37c45a74-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.831357 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.831266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43282f36-5709-462b-9d94-117b37c45a74-metrics-client-ca\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.932036 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.931995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/43282f36-5709-462b-9d94-117b37c45a74-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.932233 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.932044    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43282f36-5709-462b-9d94-117b37c45a74-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.932233 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.932075    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43282f36-5709-462b-9d94-117b37c45a74-metrics-client-ca\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.932233 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.932140    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-499sg\" (UniqueName: \"kubernetes.io/projected/43282f36-5709-462b-9d94-117b37c45a74-kube-api-access-499sg\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.933528 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.933437    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43282f36-5709-462b-9d94-117b37c45a74-metrics-client-ca\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.935265 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.935233    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/43282f36-5709-462b-9d94-117b37c45a74-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.935350 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.935314    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43282f36-5709-462b-9d94-117b37c45a74-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:55.941440 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:55.941421    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-499sg\" (UniqueName: \"kubernetes.io/projected/43282f36-5709-462b-9d94-117b37c45a74-kube-api-access-499sg\") pod \"prometheus-operator-64785b6675-fstjl\" (UID: \"43282f36-5709-462b-9d94-117b37c45a74\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:56.038504 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:56.038405    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl"
Jul 14 15:47:59.349218 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:59.349189    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8c45s"
Jul 14 15:47:59.674081 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:47:59.673609    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-fstjl"]
Jul 14 15:47:59.677649 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:47:59.677614    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43282f36_5709_462b_9d94_117b37c45a74.slice/crio-45c8500b5cc268b1779cd17f6111175affb89ec11bdb912b5b86d19d81b8868f WatchSource:0}: Error finding container 45c8500b5cc268b1779cd17f6111175affb89ec11bdb912b5b86d19d81b8868f: Status 404 returned error can't find the container with id 45c8500b5cc268b1779cd17f6111175affb89ec11bdb912b5b86d19d81b8868f
Jul 14 15:48:00.436120 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:00.436078    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8mzm5" event={"ID":"ec667542-ec92-4054-a5d5-849cfe374814","Type":"ContainerStarted","Data":"a4b3a20603989b50daac0279c1a61c73167df6c2de07e8c89383457974007a9a"}
Jul 14 15:48:00.436604 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:00.436177    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8mzm5" event={"ID":"ec667542-ec92-4054-a5d5-849cfe374814","Type":"ContainerStarted","Data":"917b230c5b516714144e96abbec219f46e5acffd9a4959ee94399b85d4cc12bd"}
Jul 14 15:48:00.436604 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:00.436211    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:48:00.438071 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:00.438034    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nwpfv" event={"ID":"e6b29cdc-5f48-48ad-b97a-42a968c2ce7d","Type":"ContainerStarted","Data":"f8290b140a727e4ba404f9d064b3299902d3b947c95715233482ca4f29d028fe"}
Jul 14 15:48:00.439320 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:00.439284    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl" event={"ID":"43282f36-5709-462b-9d94-117b37c45a74","Type":"ContainerStarted","Data":"45c8500b5cc268b1779cd17f6111175affb89ec11bdb912b5b86d19d81b8868f"}
Jul 14 15:48:00.441058 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:00.441033    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb758b489-l2g29" event={"ID":"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec","Type":"ContainerStarted","Data":"7bfdf3f1ea8813c098f93e09a34ba53f535767797d2416edfa3bde23f1b1b411"}
Jul 14 15:48:00.443047 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:00.443022    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zkc2d" event={"ID":"e8c6cfcb-efe3-4838-b2d9-614bd266711a","Type":"ContainerStarted","Data":"ed33cefd0af5ee9c8d8bfd379108d79f9e466907a2e7bebf28d76dbaa663b61b"}
Jul 14 15:48:00.443131 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:00.443070    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zkc2d" event={"ID":"e8c6cfcb-efe3-4838-b2d9-614bd266711a","Type":"ContainerStarted","Data":"7a84b4828cecbd772c8cc7172ac769e487184ac5252ea2946cde0c5231d5fb7d"}
Jul 14 15:48:00.454770 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:00.454729    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8mzm5" podStartSLOduration=1.7596493629999999 podStartE2EDuration="8.454716144s" totalImagesPullingTime="6.695066781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:00.454029381 +0000 UTC m=+63.973396802" watchObservedRunningTime="2026-07-14 15:48:00.454716144 +0000 UTC m=+63.974083597"
Jul 14 15:48:00.471583 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:00.471527    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nwpfv" podStartSLOduration=1.777093714 podStartE2EDuration="8.471508615s" totalImagesPullingTime="6.694414901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:00.470476546 +0000 UTC m=+63.989843968" watchObservedRunningTime="2026-07-14 15:48:00.471508615 +0000 UTC m=+63.990876041"
Jul 14 15:48:00.489983 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:00.489925    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fb758b489-l2g29" podStartSLOduration=2.296891431 podStartE2EDuration="8.489903187s" totalImagesPullingTime="6.193011756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:00.488885986 +0000 UTC m=+64.008253420" watchObservedRunningTime="2026-07-14 15:48:00.489903187 +0000 UTC m=+64.009270609"
Jul 14 15:48:01.398394 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.398359    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-df9ddd8bf-m2zjx"]
Jul 14 15:48:01.400958 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.400935    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.410380 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.410356    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 14 15:48:01.413092 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.413070    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-df9ddd8bf-m2zjx"]
Jul 14 15:48:01.448733 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.448687    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl" event={"ID":"43282f36-5709-462b-9d94-117b37c45a74","Type":"ContainerStarted","Data":"7cef2a501e612b333c1ba9edd86c10ffe92db20b782407825ea0c0f8f9dad245"}
Jul 14 15:48:01.449173 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.448749    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl" event={"ID":"43282f36-5709-462b-9d94-117b37c45a74","Type":"ContainerStarted","Data":"f043434ad367bc07d113c6993326da7d13a58b263bd10931878e33e2f426d36d"}
Jul 14 15:48:01.468389 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.468333    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-fstjl" podStartSLOduration=5.213447866 podStartE2EDuration="6.468315744s" totalImagesPullingTime="1.254867878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:47:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:01.467064933 +0000 UTC m=+64.986432353" watchObservedRunningTime="2026-07-14 15:48:01.468315744 +0000 UTC m=+64.987683166"
Jul 14 15:48:01.480915 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.480885    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9mvl\" (UniqueName: \"kubernetes.io/projected/e0d47717-2e66-4158-b45f-f501ac58f9b1-kube-api-access-k9mvl\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.481066 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.480940    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-config\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.481126 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.481057    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-oauth-serving-cert\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.481126 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.481089    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-service-ca\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.481126 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.481109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-oauth-config\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.481281 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.481133    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-serving-cert\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.481281 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.481189    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-trusted-ca-bundle\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.581664 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.581630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-config\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.581851 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.581827    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-oauth-serving-cert\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.581925 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.581854    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-service-ca\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.581925 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.581897    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-oauth-config\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.582017 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.581943    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-serving-cert\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.582017 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.581971    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-trusted-ca-bundle\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.582108 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.582088    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9mvl\" (UniqueName: \"kubernetes.io/projected/e0d47717-2e66-4158-b45f-f501ac58f9b1-kube-api-access-k9mvl\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.582514 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.582482    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-config\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.583176 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.583148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-oauth-serving-cert\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.583510 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.583494    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-service-ca\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.583702 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.583665    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-trusted-ca-bundle\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.586096 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.586074    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-serving-cert\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.586272 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.586256    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-oauth-config\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.592264 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.592245    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9mvl\" (UniqueName: \"kubernetes.io/projected/e0d47717-2e66-4158-b45f-f501ac58f9b1-kube-api-access-k9mvl\") pod \"console-df9ddd8bf-m2zjx\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") " pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.713730 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.713645    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:01.858469 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:01.858409    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-df9ddd8bf-m2zjx"]
Jul 14 15:48:01.863418 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:48:01.863383    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d47717_2e66_4158_b45f_f501ac58f9b1.slice/crio-3ef869865583ed89c685858bc06747e6cf0d2578ed52f1a680fb8b168c668015 WatchSource:0}: Error finding container 3ef869865583ed89c685858bc06747e6cf0d2578ed52f1a680fb8b168c668015: Status 404 returned error can't find the container with id 3ef869865583ed89c685858bc06747e6cf0d2578ed52f1a680fb8b168c668015
Jul 14 15:48:02.453927 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.453888    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df9ddd8bf-m2zjx" event={"ID":"e0d47717-2e66-4158-b45f-f501ac58f9b1","Type":"ContainerStarted","Data":"02e7944033f313906c3260efb25cc0ecae9dc333422adfba0c3a87292751b822"}
Jul 14 15:48:02.454399 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.453982    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df9ddd8bf-m2zjx" event={"ID":"e0d47717-2e66-4158-b45f-f501ac58f9b1","Type":"ContainerStarted","Data":"3ef869865583ed89c685858bc06747e6cf0d2578ed52f1a680fb8b168c668015"}
Jul 14 15:48:02.474240 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.474190    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-df9ddd8bf-m2zjx" podStartSLOduration=1.474175789 podStartE2EDuration="1.474175789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:48:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:02.473674578 +0000 UTC m=+65.993042013" watchObservedRunningTime="2026-07-14 15:48:02.474175789 +0000 UTC m=+65.993543209"
Jul 14 15:48:02.691217 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.691165    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:48:02.693698 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.693664    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 15:48:02.704194 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.704126    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bbccbea-f177-4169-a89d-a09d860543b0-metrics-certs\") pod \"network-metrics-daemon-w92wj\" (UID: \"9bbccbea-f177-4169-a89d-a09d860543b0\") " pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:48:02.756651 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.756616    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zfvj8\""
Jul 14 15:48:02.764536 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.764510    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w92wj"
Jul 14 15:48:02.792028 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.792003    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:48:02.794317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.794295    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 15:48:02.805078 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.805024    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b801942-157e-4913-98a7-44d9440b3257-original-pull-secret\") pod \"global-pull-secret-syncer-8674z\" (UID: \"0b801942-157e-4913-98a7-44d9440b3257\") " pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:48:02.892416 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.892383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxj6\" (UniqueName: \"kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6\") pod \"network-check-target-jxlsn\" (UID: \"72ca74e2-c3b6-42e4-98f2-f12330a7bb48\") " pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:48:02.895016 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.894991    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 15:48:02.905659 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.905634    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 15:48:02.916467 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:02.916420    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hxj6\" (UniqueName: \"kubernetes.io/projected/72ca74e2-c3b6-42e4-98f2-f12330a7bb48-kube-api-access-5hxj6\") pod \"network-check-target-jxlsn\" (UID: \"72ca74e2-c3b6-42e4-98f2-f12330a7bb48\") " pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:48:03.044308 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.044281    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8674z"
Jul 14 15:48:03.064950 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.064884    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gt8h7\""
Jul 14 15:48:03.072552 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.072525    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"]
Jul 14 15:48:03.072675 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.072533    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:48:03.074794 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.074691    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.080812 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.080783    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 14 15:48:03.080930 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.080916    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 14 15:48:03.081434 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.081404    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-v5jll\""
Jul 14 15:48:03.091225 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.091190    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"]
Jul 14 15:48:03.099488 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.099326    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-prfgb"]
Jul 14 15:48:03.105502 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.105147    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.108727 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.108664    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 14 15:48:03.110077 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.108973    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bchnj\""
Jul 14 15:48:03.111308 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.111284    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-g8z6p"]
Jul 14 15:48:03.112326 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.112018    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 14 15:48:03.112326 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.112169    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 14 15:48:03.114752 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.114734    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-prfgb"]
Jul 14 15:48:03.115050 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.114960    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.121107 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.121089    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 14 15:48:03.122093 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.122022    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 14 15:48:03.122437 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.122419    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 14 15:48:03.122823 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.122805    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vsgwq\""
Jul 14 15:48:03.185090 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.185063    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:48:03.185320 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.185297    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:48:03.186390 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.186360    2560 patch_prober.go:28] interesting pod/console-5fb758b489-l2g29 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body=
Jul 14 15:48:03.186527 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.186415    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5fb758b489-l2g29" podUID="e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused"
Jul 14 15:48:03.188582 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.188561    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w92wj"]
Jul 14 15:48:03.191775 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:48:03.191747    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bbccbea_f177_4169_a89d_a09d860543b0.slice/crio-8c258abf2cbe9e36a973ed4d3acbc2c64df9ffa0055d6f2a9522cd3432670d56 WatchSource:0}: Error finding container 8c258abf2cbe9e36a973ed4d3acbc2c64df9ffa0055d6f2a9522cd3432670d56: Status 404 returned error can't find the container with id 8c258abf2cbe9e36a973ed4d3acbc2c64df9ffa0055d6f2a9522cd3432670d56
Jul 14 15:48:03.199569 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199549    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c827943c-e597-4cd3-8964-a55060096e6d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.199680 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199583    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-textfile\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.199680 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199612    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8586edde-9643-4c5f-932f-0597d5afd316-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.199680 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199641    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8586edde-9643-4c5f-932f-0597d5afd316-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.199680 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199666    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt54v\" (UniqueName: \"kubernetes.io/projected/c827943c-e597-4cd3-8964-a55060096e6d-kube-api-access-zt54v\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.199885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199694    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c827943c-e597-4cd3-8964-a55060096e6d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.199885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199750    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c827943c-e597-4cd3-8964-a55060096e6d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.199885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199776    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ee18d4c-17a6-424d-9a79-5e01177de242-root\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.199885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199807    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqvtp\" (UniqueName: \"kubernetes.io/projected/8ee18d4c-17a6-424d-9a79-5e01177de242-kube-api-access-gqvtp\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.199885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199833    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c827943c-e597-4cd3-8964-a55060096e6d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.199885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199871    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ee18d4c-17a6-424d-9a79-5e01177de242-metrics-client-ca\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.200166 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199896    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqkwj\" (UniqueName: \"kubernetes.io/projected/8586edde-9643-4c5f-932f-0597d5afd316-kube-api-access-hqkwj\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.200166 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199923    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c827943c-e597-4cd3-8964-a55060096e6d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.200166 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199952    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8586edde-9643-4c5f-932f-0597d5afd316-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.200166 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.199982    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ee18d4c-17a6-424d-9a79-5e01177de242-sys\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.200166 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.200018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-tls\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.200166 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.200043    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.200166 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.200074    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-wtmp\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.200166 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.200099    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-accelerators-collector-config\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.217252 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.217222    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8674z"]
Jul 14 15:48:03.220171 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:48:03.220131    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b801942_157e_4913_98a7_44d9440b3257.slice/crio-a3b6e58231f52b762613f536c5cce653bdaf67ac1e0eed5642a9a91f010825f6 WatchSource:0}: Error finding container a3b6e58231f52b762613f536c5cce653bdaf67ac1e0eed5642a9a91f010825f6: Status 404 returned error can't find the container with id a3b6e58231f52b762613f536c5cce653bdaf67ac1e0eed5642a9a91f010825f6
Jul 14 15:48:03.246272 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.246245    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jxlsn"]
Jul 14 15:48:03.251625 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:48:03.251604    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ca74e2_c3b6_42e4_98f2_f12330a7bb48.slice/crio-29bd58a5f2dbdc8dab98da18a7bdab8e7b8701922749b642dcd526011c4a5a57 WatchSource:0}: Error finding container 29bd58a5f2dbdc8dab98da18a7bdab8e7b8701922749b642dcd526011c4a5a57: Status 404 returned error can't find the container with id 29bd58a5f2dbdc8dab98da18a7bdab8e7b8701922749b642dcd526011c4a5a57
Jul 14 15:48:03.301357 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301301    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c827943c-e597-4cd3-8964-a55060096e6d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.301357 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301346    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-textfile\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.301516 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301378    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8586edde-9643-4c5f-932f-0597d5afd316-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.301516 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301410    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8586edde-9643-4c5f-932f-0597d5afd316-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.301516 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301438    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt54v\" (UniqueName: \"kubernetes.io/projected/c827943c-e597-4cd3-8964-a55060096e6d-kube-api-access-zt54v\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.301516 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301494    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c827943c-e597-4cd3-8964-a55060096e6d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.301714 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301542    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c827943c-e597-4cd3-8964-a55060096e6d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.301714 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301571    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ee18d4c-17a6-424d-9a79-5e01177de242-root\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.301714 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301627    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ee18d4c-17a6-424d-9a79-5e01177de242-root\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.301865 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301723    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-textfile\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.301865 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301732    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c827943c-e597-4cd3-8964-a55060096e6d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.301865 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301785    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqvtp\" (UniqueName: \"kubernetes.io/projected/8ee18d4c-17a6-424d-9a79-5e01177de242-kube-api-access-gqvtp\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.301865 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301818    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c827943c-e597-4cd3-8964-a55060096e6d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.302062 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301872    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ee18d4c-17a6-424d-9a79-5e01177de242-metrics-client-ca\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.302062 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.301899    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqkwj\" (UniqueName: \"kubernetes.io/projected/8586edde-9643-4c5f-932f-0597d5afd316-kube-api-access-hqkwj\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.302184 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302158    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8586edde-9643-4c5f-932f-0597d5afd316-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.302245 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302171    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c827943c-e597-4cd3-8964-a55060096e6d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.302245 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302232    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c827943c-e597-4cd3-8964-a55060096e6d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.302345 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302248    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8586edde-9643-4c5f-932f-0597d5afd316-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.302345 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302294    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ee18d4c-17a6-424d-9a79-5e01177de242-sys\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.302345 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302335    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-tls\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.302511 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302361    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.302511 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302390    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-wtmp\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.302511 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302413    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-accelerators-collector-config\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.302511 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302440    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c827943c-e597-4cd3-8964-a55060096e6d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.302697 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302603    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ee18d4c-17a6-424d-9a79-5e01177de242-metrics-client-ca\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.302697 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302626    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ee18d4c-17a6-424d-9a79-5e01177de242-sys\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.302864 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.302727    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-wtmp\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.303274 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.303168    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-accelerators-collector-config\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.304811 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.304792    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c827943c-e597-4cd3-8964-a55060096e6d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.304894 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.304810    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-tls\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.305065 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.305042    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8586edde-9643-4c5f-932f-0597d5afd316-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.305126 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.305104    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ee18d4c-17a6-424d-9a79-5e01177de242-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.305164 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.305112    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c827943c-e597-4cd3-8964-a55060096e6d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.305225 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.305210    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8586edde-9643-4c5f-932f-0597d5afd316-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.309662 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.309642    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt54v\" (UniqueName: \"kubernetes.io/projected/c827943c-e597-4cd3-8964-a55060096e6d-kube-api-access-zt54v\") pod \"kube-state-metrics-785778ff74-prfgb\" (UID: \"c827943c-e597-4cd3-8964-a55060096e6d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.309831 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.309809    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqvtp\" (UniqueName: \"kubernetes.io/projected/8ee18d4c-17a6-424d-9a79-5e01177de242-kube-api-access-gqvtp\") pod \"node-exporter-g8z6p\" (UID: \"8ee18d4c-17a6-424d-9a79-5e01177de242\") " pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.310541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.310526    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqkwj\" (UniqueName: \"kubernetes.io/projected/8586edde-9643-4c5f-932f-0597d5afd316-kube-api-access-hqkwj\") pod \"openshift-state-metrics-84bd68b896-bzc2t\" (UID: \"8586edde-9643-4c5f-932f-0597d5afd316\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.398855 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.398824    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"
Jul 14 15:48:03.437245 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.437208    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb"
Jul 14 15:48:03.442981 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.442959    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g8z6p"
Jul 14 15:48:03.453815 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:48:03.453782    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee18d4c_17a6_424d_9a79_5e01177de242.slice/crio-74ff2e4beaa35e0a47ebb7b194ff8450e70f74912f3d3c899a7f1f869ec791d0 WatchSource:0}: Error finding container 74ff2e4beaa35e0a47ebb7b194ff8450e70f74912f3d3c899a7f1f869ec791d0: Status 404 returned error can't find the container with id 74ff2e4beaa35e0a47ebb7b194ff8450e70f74912f3d3c899a7f1f869ec791d0
Jul 14 15:48:03.460239 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.460209    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8674z" event={"ID":"0b801942-157e-4913-98a7-44d9440b3257","Type":"ContainerStarted","Data":"a3b6e58231f52b762613f536c5cce653bdaf67ac1e0eed5642a9a91f010825f6"}
Jul 14 15:48:03.462189 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.462162    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w92wj" event={"ID":"9bbccbea-f177-4169-a89d-a09d860543b0","Type":"ContainerStarted","Data":"8c258abf2cbe9e36a973ed4d3acbc2c64df9ffa0055d6f2a9522cd3432670d56"}
Jul 14 15:48:03.464791 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.464771    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zkc2d" event={"ID":"e8c6cfcb-efe3-4838-b2d9-614bd266711a","Type":"ContainerStarted","Data":"bde99c54dc258e8ff22c0be69456831001137eb9d62b17a5647baedd3933f7ef"}
Jul 14 15:48:03.466131 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.466105    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g8z6p" event={"ID":"8ee18d4c-17a6-424d-9a79-5e01177de242","Type":"ContainerStarted","Data":"74ff2e4beaa35e0a47ebb7b194ff8450e70f74912f3d3c899a7f1f869ec791d0"}
Jul 14 15:48:03.467361 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.467337    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jxlsn" event={"ID":"72ca74e2-c3b6-42e4-98f2-f12330a7bb48","Type":"ContainerStarted","Data":"29bd58a5f2dbdc8dab98da18a7bdab8e7b8701922749b642dcd526011c4a5a57"}
Jul 14 15:48:03.487984 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.487934    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zkc2d" podStartSLOduration=1.555919295 podStartE2EDuration="11.487913304s" totalImagesPullingTime="9.931994009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:47:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:03.487558151 +0000 UTC m=+67.006925571" watchObservedRunningTime="2026-07-14 15:48:03.487913304 +0000 UTC m=+67.007280725"
Jul 14 15:48:03.538318 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.538288    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t"]
Jul 14 15:48:03.546555 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:48:03.546528    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8586edde_9643_4c5f_932f_0597d5afd316.slice/crio-2c357c26f58ea9123b036f7e05985dcba379fc35ee57dfba76bfd49cc2b015d8 WatchSource:0}: Error finding container 2c357c26f58ea9123b036f7e05985dcba379fc35ee57dfba76bfd49cc2b015d8: Status 404 returned error can't find the container with id 2c357c26f58ea9123b036f7e05985dcba379fc35ee57dfba76bfd49cc2b015d8
Jul 14 15:48:03.577549 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:03.577511    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-prfgb"]
Jul 14 15:48:03.579874 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:48:03.579851    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc827943c_e597_4cd3_8964_a55060096e6d.slice/crio-f2edb52962ba2aefb870f24d6079d65ae3cd518911992eb4ea88426c1f596565 WatchSource:0}: Error finding container f2edb52962ba2aefb870f24d6079d65ae3cd518911992eb4ea88426c1f596565: Status 404 returned error can't find the container with id f2edb52962ba2aefb870f24d6079d65ae3cd518911992eb4ea88426c1f596565
Jul 14 15:48:04.177902 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.177865    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 15:48:04.182959 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.182865    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.185743 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.185582    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 14 15:48:04.186646 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.186625    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 14 15:48:04.186750 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.186686    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-njrlw\""
Jul 14 15:48:04.187005 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.186938    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 14 15:48:04.187005 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.186943    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 14 15:48:04.187173 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.187007    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 14 15:48:04.187251 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.187234    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 14 15:48:04.187290 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.187249    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 14 15:48:04.187592 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.187576    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 14 15:48:04.187668 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.187635    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 14 15:48:04.204476 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.204382    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 15:48:04.309088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.308685    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.308738    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.308789    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.308820    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.308846    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-volume\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.308869    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-out\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.308893    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsm6f\" (UniqueName: \"kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-kube-api-access-jsm6f\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.308932    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-web-config\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.308961    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.308996    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.309021    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309760 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.309113    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.309760 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.309148    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410479 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410640 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410488    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410640 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410526    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410640 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410556    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410640 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410589    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410640 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410617    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410802 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410646    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-volume\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410802 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410674    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-out\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410802 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410708    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsm6f\" (UniqueName: \"kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-kube-api-access-jsm6f\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410802 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410748    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-web-config\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410802 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410775    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410974 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410809    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.410974 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.410837    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.414160 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.413303    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.414160 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.414107    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.414718 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.414663    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.416518 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.416418    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.417789 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.417683    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.417789 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.417757    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-volume\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.419338 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.419313    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.421288 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.421266    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-out\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.421663 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.421625    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-web-config\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.422405 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.422360    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.423678 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.423655    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.424866 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.424826    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsm6f\" (UniqueName: \"kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-kube-api-access-jsm6f\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.426100 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.426062    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:04.482974 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.482892    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb" event={"ID":"c827943c-e597-4cd3-8964-a55060096e6d","Type":"ContainerStarted","Data":"f2edb52962ba2aefb870f24d6079d65ae3cd518911992eb4ea88426c1f596565"}
Jul 14 15:48:04.486410 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.486379    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t" event={"ID":"8586edde-9643-4c5f-932f-0597d5afd316","Type":"ContainerStarted","Data":"3dd8aa4b17f42f046ae5d4794d2ec30e82dd7f67eb6a6d9bc988bbd47054d501"}
Jul 14 15:48:04.486566 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.486441    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t" event={"ID":"8586edde-9643-4c5f-932f-0597d5afd316","Type":"ContainerStarted","Data":"b0cd47c81c6363a572bff1095e6a4c97c49e16985a53bad487561b50c48bac35"}
Jul 14 15:48:04.486566 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.486497    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t" event={"ID":"8586edde-9643-4c5f-932f-0597d5afd316","Type":"ContainerStarted","Data":"2c357c26f58ea9123b036f7e05985dcba379fc35ee57dfba76bfd49cc2b015d8"}
Jul 14 15:48:04.493897 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:04.493870    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:48:05.162526 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.162494    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5965d957f-58mld"]
Jul 14 15:48:05.165334 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.165313    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.170383 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.170345    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 14 15:48:05.170523 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.170402    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 14 15:48:05.170523 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.170461    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-64ms98hd609ma\""
Jul 14 15:48:05.170672 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.170646    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 14 15:48:05.171153 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.170971    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ftck7\""
Jul 14 15:48:05.171490 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.171473    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 14 15:48:05.175169 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.175148    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 14 15:48:05.197293 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.197268    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5965d957f-58mld"]
Jul 14 15:48:05.320473 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.320156    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.320473 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.320196    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.320473 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.320223    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.320473 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.320249    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.320473 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.320274    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-grpc-tls\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.320473 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.320322    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/812da01e-0d77-451a-8942-eb32e0f0728a-metrics-client-ca\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.320473 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.320355    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-tls\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.320473 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.320414    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5bjj\" (UniqueName: \"kubernetes.io/projected/812da01e-0d77-451a-8942-eb32e0f0728a-kube-api-access-r5bjj\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.413870 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.413704    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 15:48:05.424500 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.421651    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.424500 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.421696    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.424500 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.421726    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.424500 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.421753    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.424500 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.421778    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-grpc-tls\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.424500 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.421831    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/812da01e-0d77-451a-8942-eb32e0f0728a-metrics-client-ca\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.424500 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.421868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-tls\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.424500 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.421931    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5bjj\" (UniqueName: \"kubernetes.io/projected/812da01e-0d77-451a-8942-eb32e0f0728a-kube-api-access-r5bjj\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.426855 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.426259    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/812da01e-0d77-451a-8942-eb32e0f0728a-metrics-client-ca\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.429685 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.429297    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-grpc-tls\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.430264 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.430222    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.430843 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.430794    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.434247 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.434103    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-tls\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.434764 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.434697    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.437325 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.435664    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/812da01e-0d77-451a-8942-eb32e0f0728a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.458130 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.458095    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5bjj\" (UniqueName: \"kubernetes.io/projected/812da01e-0d77-451a-8942-eb32e0f0728a-kube-api-access-r5bjj\") pod \"thanos-querier-5965d957f-58mld\" (UID: \"812da01e-0d77-451a-8942-eb32e0f0728a\") " pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.478991 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.478899    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:05.495332 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.495123    2560 generic.go:356] "Generic (PLEG): container finished" podID="8ee18d4c-17a6-424d-9a79-5e01177de242" containerID="ecc0e78f7b1e89bd54e00f1bc7ee298a22eceff6f636aeb454723293b22abe78" exitCode=0
Jul 14 15:48:05.495332 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.495237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g8z6p" event={"ID":"8ee18d4c-17a6-424d-9a79-5e01177de242","Type":"ContainerDied","Data":"ecc0e78f7b1e89bd54e00f1bc7ee298a22eceff6f636aeb454723293b22abe78"}
Jul 14 15:48:05.499962 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.499149    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t" event={"ID":"8586edde-9643-4c5f-932f-0597d5afd316","Type":"ContainerStarted","Data":"d3b5f226d0a88e3a8eaed7504c32b3f1586101c41f9cb69c9d1289403881d710"}
Jul 14 15:48:05.503126 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.501981    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerStarted","Data":"0abc21635cc5f3e74fe420e6668bae3c27438c81e481937b9ca895cfb28eb17a"}
Jul 14 15:48:05.505419 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.505261    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb" event={"ID":"c827943c-e597-4cd3-8964-a55060096e6d","Type":"ContainerStarted","Data":"68b63bbaecb6ce8512b094758c6dd01926adfb3d38d192b44ca96991765c70ad"}
Jul 14 15:48:05.505419 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.505322    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb" event={"ID":"c827943c-e597-4cd3-8964-a55060096e6d","Type":"ContainerStarted","Data":"3a01001a1d93a45262a93226bbd026de50c7b4e5d9c18de2e2d234b7faf50344"}
Jul 14 15:48:05.568997 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.568544    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bzc2t" podStartSLOduration=1.029588957 podStartE2EDuration="2.568521605s" totalImagesPullingTime="1.538932648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:48:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:05.566491731 +0000 UTC m=+69.085859153" watchObservedRunningTime="2026-07-14 15:48:05.568521605 +0000 UTC m=+69.087889026"
Jul 14 15:48:05.707944 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:05.707849    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5965d957f-58mld"]
Jul 14 15:48:05.713281 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:48:05.713250    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812da01e_0d77_451a_8942_eb32e0f0728a.slice/crio-c616ce98b42cddeed987c89516d6c5f2478acf45a1515c3f307e3f622e3c383f WatchSource:0}: Error finding container c616ce98b42cddeed987c89516d6c5f2478acf45a1515c3f307e3f622e3c383f: Status 404 returned error can't find the container with id c616ce98b42cddeed987c89516d6c5f2478acf45a1515c3f307e3f622e3c383f
Jul 14 15:48:06.521642 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:06.521563    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb" event={"ID":"c827943c-e597-4cd3-8964-a55060096e6d","Type":"ContainerStarted","Data":"32578eb5615e5225d6bfaf2f195d20db97ea6e656a269252a2320789cd3c9881"}
Jul 14 15:48:06.526671 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:06.526052    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g8z6p" event={"ID":"8ee18d4c-17a6-424d-9a79-5e01177de242","Type":"ContainerStarted","Data":"d6e730aa904b6264add3055c9349ed4673259a047272084fb4fdc98798633642"}
Jul 14 15:48:06.526671 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:06.526106    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g8z6p" event={"ID":"8ee18d4c-17a6-424d-9a79-5e01177de242","Type":"ContainerStarted","Data":"2ae6057b740e01be626378cb08aeb4c148cc31bcf33f7c126358c7294095c8c4"}
Jul 14 15:48:06.529193 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:06.529160    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5965d957f-58mld" event={"ID":"812da01e-0d77-451a-8942-eb32e0f0728a","Type":"ContainerStarted","Data":"c616ce98b42cddeed987c89516d6c5f2478acf45a1515c3f307e3f622e3c383f"}
Jul 14 15:48:06.550561 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:06.549190    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-prfgb" podStartSLOduration=1.920486156 podStartE2EDuration="3.549170831s" totalImagesPullingTime="1.628684675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:48:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:06.547739179 +0000 UTC m=+70.067106631" watchObservedRunningTime="2026-07-14 15:48:06.549170831 +0000 UTC m=+70.068538260"
Jul 14 15:48:06.578084 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:06.577326    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-g8z6p" podStartSLOduration=1.699991481 podStartE2EDuration="3.577304209s" totalImagesPullingTime="1.749610032s" totalInitContainerRuntime="127.702696ms" isStatefulPod=true podCreationTimestamp="2026-07-14 15:48:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:06.575423116 +0000 UTC m=+70.094790546" watchObservedRunningTime="2026-07-14 15:48:06.577304209 +0000 UTC m=+70.096671634"
Jul 14 15:48:07.472181 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.472139    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"]
Jul 14 15:48:07.474530 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.474506    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.480277 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.480250    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 14 15:48:07.480423 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.480388    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 14 15:48:07.480620 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.480598    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 14 15:48:07.481281 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.481261    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 14 15:48:07.481401 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.481357    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-rmckl\""
Jul 14 15:48:07.481727 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.481515    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4qspbmqcf6r4l\""
Jul 14 15:48:07.490056 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.490037    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"]
Jul 14 15:48:07.539380 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.539269    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerStarted","Data":"025f9ab81814d14278fd4cd98c51a0a24ac6d83cef13243b00aa593fa8340b13"}
Jul 14 15:48:07.541556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.541525    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w92wj" event={"ID":"9bbccbea-f177-4169-a89d-a09d860543b0","Type":"ContainerStarted","Data":"b00f2b6c61623a93ec32b76530b6c988826c540c0a4091fd1864216885008147"}
Jul 14 15:48:07.644562 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.644527    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ee471a45-f22c-4773-96b7-c96d1da35cee-secret-metrics-server-tls\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.644746 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.644608    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ee471a45-f22c-4773-96b7-c96d1da35cee-secret-metrics-server-client-certs\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.644746 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.644638    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee471a45-f22c-4773-96b7-c96d1da35cee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.644746 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.644683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdf4x\" (UniqueName: \"kubernetes.io/projected/ee471a45-f22c-4773-96b7-c96d1da35cee-kube-api-access-hdf4x\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.644746 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.644717    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee471a45-f22c-4773-96b7-c96d1da35cee-client-ca-bundle\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.644904 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.644744    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ee471a45-f22c-4773-96b7-c96d1da35cee-metrics-server-audit-profiles\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.644904 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.644768    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ee471a45-f22c-4773-96b7-c96d1da35cee-audit-log\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.745768 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.745681    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ee471a45-f22c-4773-96b7-c96d1da35cee-secret-metrics-server-tls\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.745768 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.745752    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ee471a45-f22c-4773-96b7-c96d1da35cee-secret-metrics-server-client-certs\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.745986 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.745780    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee471a45-f22c-4773-96b7-c96d1da35cee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.745986 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.745813    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdf4x\" (UniqueName: \"kubernetes.io/projected/ee471a45-f22c-4773-96b7-c96d1da35cee-kube-api-access-hdf4x\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.745986 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.745868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee471a45-f22c-4773-96b7-c96d1da35cee-client-ca-bundle\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.745986 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.745897    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ee471a45-f22c-4773-96b7-c96d1da35cee-metrics-server-audit-profiles\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.745986 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.745922    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ee471a45-f22c-4773-96b7-c96d1da35cee-audit-log\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.746415 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.746399    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ee471a45-f22c-4773-96b7-c96d1da35cee-audit-log\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.746979 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.746952    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee471a45-f22c-4773-96b7-c96d1da35cee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.747353 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.747330    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ee471a45-f22c-4773-96b7-c96d1da35cee-metrics-server-audit-profiles\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.749556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.749514    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ee471a45-f22c-4773-96b7-c96d1da35cee-secret-metrics-server-tls\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.750080 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.750041    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee471a45-f22c-4773-96b7-c96d1da35cee-client-ca-bundle\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.750173 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.750131    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ee471a45-f22c-4773-96b7-c96d1da35cee-secret-metrics-server-client-certs\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.756464 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.756417    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdf4x\" (UniqueName: \"kubernetes.io/projected/ee471a45-f22c-4773-96b7-c96d1da35cee-kube-api-access-hdf4x\") pod \"metrics-server-5c464d7cf4-4w7b2\" (UID: \"ee471a45-f22c-4773-96b7-c96d1da35cee\") " pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.786329 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.786297    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:07.861465 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.861402    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"]
Jul 14 15:48:07.879182 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.879140    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"]
Jul 14 15:48:07.879353 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.879296    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"
Jul 14 15:48:07.882633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.882379    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 14 15:48:07.882633 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:07.882474    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-c5f6h\""
Jul 14 15:48:08.048277 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:08.048198    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kgzg\" (UniqueName: \"kubernetes.io/projected/79315669-ca17-45a4-9bb9-303e739eb8dd-kube-api-access-8kgzg\") pod \"monitoring-plugin-5f68df7f58-9f57r\" (UID: \"79315669-ca17-45a4-9bb9-303e739eb8dd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"
Jul 14 15:48:08.048494 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:08.048356    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/79315669-ca17-45a4-9bb9-303e739eb8dd-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-9f57r\" (UID: \"79315669-ca17-45a4-9bb9-303e739eb8dd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"
Jul 14 15:48:08.149797 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:08.149772    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/79315669-ca17-45a4-9bb9-303e739eb8dd-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-9f57r\" (UID: \"79315669-ca17-45a4-9bb9-303e739eb8dd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"
Jul 14 15:48:08.149951 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:08.149830    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kgzg\" (UniqueName: \"kubernetes.io/projected/79315669-ca17-45a4-9bb9-303e739eb8dd-kube-api-access-8kgzg\") pod \"monitoring-plugin-5f68df7f58-9f57r\" (UID: \"79315669-ca17-45a4-9bb9-303e739eb8dd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"
Jul 14 15:48:08.150016 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:48:08.149953    2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 14 15:48:08.150071 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:48:08.150030    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79315669-ca17-45a4-9bb9-303e739eb8dd-monitoring-plugin-cert podName:79315669-ca17-45a4-9bb9-303e739eb8dd nodeName:}" failed. No retries permitted until 2026-07-14 15:48:08.650009604 +0000 UTC m=+72.169377027 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/79315669-ca17-45a4-9bb9-303e739eb8dd-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-9f57r" (UID: "79315669-ca17-45a4-9bb9-303e739eb8dd") : secret "monitoring-plugin-cert" not found
Jul 14 15:48:08.160717 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:08.160688    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kgzg\" (UniqueName: \"kubernetes.io/projected/79315669-ca17-45a4-9bb9-303e739eb8dd-kube-api-access-8kgzg\") pod \"monitoring-plugin-5f68df7f58-9f57r\" (UID: \"79315669-ca17-45a4-9bb9-303e739eb8dd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"
Jul 14 15:48:08.547031 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:08.546995    2560 generic.go:356] "Generic (PLEG): container finished" podID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerID="025f9ab81814d14278fd4cd98c51a0a24ac6d83cef13243b00aa593fa8340b13" exitCode=0
Jul 14 15:48:08.547541 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:08.547082    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerDied","Data":"025f9ab81814d14278fd4cd98c51a0a24ac6d83cef13243b00aa593fa8340b13"}
Jul 14 15:48:08.654196 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:08.654164    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/79315669-ca17-45a4-9bb9-303e739eb8dd-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-9f57r\" (UID: \"79315669-ca17-45a4-9bb9-303e739eb8dd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"
Jul 14 15:48:08.657269 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:08.657243    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/79315669-ca17-45a4-9bb9-303e739eb8dd-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-9f57r\" (UID: \"79315669-ca17-45a4-9bb9-303e739eb8dd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"
Jul 14 15:48:08.793668 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:08.793631    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"
Jul 14 15:48:09.539973 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.539934    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 15:48:09.544038 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.543986    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.548298 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.548275    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 14 15:48:09.548728 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.548641    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 14 15:48:09.548728 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.548354    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 14 15:48:09.548728 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.548379    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 14 15:48:09.548728 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.548581    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mhrsz\""
Jul 14 15:48:09.549065 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.549045    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 14 15:48:09.549150 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.549062    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 14 15:48:09.549223 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.549200    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 14 15:48:09.549739 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.549719    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 14 15:48:09.550066 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.549931    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 14 15:48:09.550066 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.549941    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 14 15:48:09.550066 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.549963    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 14 15:48:09.550066 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.549980    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 14 15:48:09.550066 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.550000    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5b6sg8bv5721m\""
Jul 14 15:48:09.550577 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.550390    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 14 15:48:09.554787 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.554766    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 14 15:48:09.565907 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.565881    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 15:48:09.665282 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665236    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665282 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665277    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665297    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf7g2\" (UniqueName: \"kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-kube-api-access-xf7g2\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665324    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665395    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665436    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-web-config\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665529    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665817 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665817 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665604    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665817 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665633    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665817 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665654    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665817 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665673    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665817 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665705    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665817 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665729    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-config\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665817 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.665817 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665769    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.666167 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665840    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.666167 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665867    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-config-out\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.666167 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.665883    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767054 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767014    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-config-out\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767054 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767067    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767326 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767131    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767326 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767326 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767296    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf7g2\" (UniqueName: \"kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-kube-api-access-xf7g2\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767434 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767352    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767434 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767399    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767434 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767425    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767589 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767485    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-web-config\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767589 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767522    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767589 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767538    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767589 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767585    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767775 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767775 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767643    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767775 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767775 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767715    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767775 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767739    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-config\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.767775 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767762    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.770185 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767790    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.770185 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767825    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.770185 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.767846    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.770185 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.768481    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.770185 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.769081    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.770185 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.769959    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.772346 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.770927    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-config-out\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.772346 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.771253    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.772346 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.771296    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.772346 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.771506    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.772346 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.771729    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.772346 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.771737    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.772734 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.772532    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-config\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.773199 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.773174    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.773319 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.773242    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.773980 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.773957    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.774476 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.774432    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.774830 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.774803    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.775210 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.775169    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-web-config\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.775462 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.775431    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.775809 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.775787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.777502 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.777478    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf7g2\" (UniqueName: \"kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-kube-api-access-xf7g2\") pod \"prometheus-k8s-0\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:09.862347 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:09.862065    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:10.240703 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:10.240613    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fb758b489-l2g29"]
Jul 14 15:48:10.451274 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:10.451247    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8mzm5"
Jul 14 15:48:10.848334 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:10.847978    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"]
Jul 14 15:48:10.899050 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:10.898930    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 15:48:10.906476 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:48:10.905609    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38c5d7de_be40_47f8_ba10_02f952515179.slice/crio-5d9dfd54d4b6947f3be94f8714b5e5e4f98197cdf14d8985dedd854f400d2923 WatchSource:0}: Error finding container 5d9dfd54d4b6947f3be94f8714b5e5e4f98197cdf14d8985dedd854f400d2923: Status 404 returned error can't find the container with id 5d9dfd54d4b6947f3be94f8714b5e5e4f98197cdf14d8985dedd854f400d2923
Jul 14 15:48:10.910435 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:10.910401    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"]
Jul 14 15:48:10.912701 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:48:10.912640    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79315669_ca17_45a4_9bb9_303e739eb8dd.slice/crio-9e277ea71e09032d7e12b5e2ccd409a1757ebacd9bc3009bf6a76d14201a47f4 WatchSource:0}: Error finding container 9e277ea71e09032d7e12b5e2ccd409a1757ebacd9bc3009bf6a76d14201a47f4: Status 404 returned error can't find the container with id 9e277ea71e09032d7e12b5e2ccd409a1757ebacd9bc3009bf6a76d14201a47f4
Jul 14 15:48:11.565321 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.565285    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jxlsn" event={"ID":"72ca74e2-c3b6-42e4-98f2-f12330a7bb48","Type":"ContainerStarted","Data":"8751108e075ca819ae636a2ed7b1a60649de6c693af00192a446bac94813b128"}
Jul 14 15:48:11.565533 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.565377    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:48:11.566769 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.566717    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r" event={"ID":"79315669-ca17-45a4-9bb9-303e739eb8dd","Type":"ContainerStarted","Data":"9e277ea71e09032d7e12b5e2ccd409a1757ebacd9bc3009bf6a76d14201a47f4"}
Jul 14 15:48:11.568221 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.568193    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8674z" event={"ID":"0b801942-157e-4913-98a7-44d9440b3257","Type":"ContainerStarted","Data":"8313a44af29943d44db09b3a2399094ba40448dc56442924db2e69386066fa64"}
Jul 14 15:48:11.570632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.570608    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5965d957f-58mld" event={"ID":"812da01e-0d77-451a-8942-eb32e0f0728a","Type":"ContainerStarted","Data":"8103407e158e060e4129907e50ecb98b563d1c7bccaad58b2101f236d2df69d7"}
Jul 14 15:48:11.570743 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.570675    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5965d957f-58mld" event={"ID":"812da01e-0d77-451a-8942-eb32e0f0728a","Type":"ContainerStarted","Data":"d88a0347ee97b7ec64e81d25ec81ee82942297ddc7ff86a105cbde6abf44d26e"}
Jul 14 15:48:11.570743 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.570718    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5965d957f-58mld" event={"ID":"812da01e-0d77-451a-8942-eb32e0f0728a","Type":"ContainerStarted","Data":"2d87c5b723775b545c84eedb150ae1d90d57488ec13e8d6e0ff929240426d76c"}
Jul 14 15:48:11.572501 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.572476    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w92wj" event={"ID":"9bbccbea-f177-4169-a89d-a09d860543b0","Type":"ContainerStarted","Data":"ebed9636664cc20d1f3eac5713a05d78cae4cce7b1d8b4d1e41121ed1be0b3ba"}
Jul 14 15:48:11.574385 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.574360    2560 generic.go:356] "Generic (PLEG): container finished" podID="38c5d7de-be40-47f8-ba10-02f952515179" containerID="602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64" exitCode=0
Jul 14 15:48:11.574499 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.574434    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerDied","Data":"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"}
Jul 14 15:48:11.574569 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.574511    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerStarted","Data":"5d9dfd54d4b6947f3be94f8714b5e5e4f98197cdf14d8985dedd854f400d2923"}
Jul 14 15:48:11.575919 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.575884    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2" event={"ID":"ee471a45-f22c-4773-96b7-c96d1da35cee","Type":"ContainerStarted","Data":"6e8bede494be9864d481acdac5af8de23167588fddf64e98842daf7e1a27a135"}
Jul 14 15:48:11.583348 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.583280    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jxlsn" podStartSLOduration=66.174675953 podStartE2EDuration="1m13.583261998s" totalImagesPullingTime="7.408586045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:11.582296703 +0000 UTC m=+75.101664128" watchObservedRunningTime="2026-07-14 15:48:11.583261998 +0000 UTC m=+75.102629416"
Jul 14 15:48:11.640549 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.640493    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8674z" podStartSLOduration=66.192914132 podStartE2EDuration="1m13.64047671s" totalImagesPullingTime="7.447562578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 15:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:11.629384069 +0000 UTC m=+75.148751491" watchObservedRunningTime="2026-07-14 15:48:11.64047671 +0000 UTC m=+75.159844132"
Jul 14 15:48:11.655079 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.655019    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w92wj" podStartSLOduration=71.14534823299999 podStartE2EDuration="1m14.655003217s" totalImagesPullingTime="3.509654984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:11.654500175 +0000 UTC m=+75.173867599" watchObservedRunningTime="2026-07-14 15:48:11.655003217 +0000 UTC m=+75.174370638"
Jul 14 15:48:11.713903 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.713851    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:11.714074 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.713963    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:11.715577 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.715240    2560 patch_prober.go:28] interesting pod/console-df9ddd8bf-m2zjx container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body=
Jul 14 15:48:11.715577 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:11.715291    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-df9ddd8bf-m2zjx" podUID="e0d47717-2e66-4158-b45f-f501ac58f9b1" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused"
Jul 14 15:48:13.587422 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.587333    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r" event={"ID":"79315669-ca17-45a4-9bb9-303e739eb8dd","Type":"ContainerStarted","Data":"725f10ef0e69c77e0c6e64939a713edb601cc74de3995007cd402d8fef15040a"}
Jul 14 15:48:13.587794 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.587504    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"
Jul 14 15:48:13.593431 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.592518    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerStarted","Data":"89a853ba37d788afa5d6d9dd3c906ce0d3d52cca549fc62333cbf9408f481aca"}
Jul 14 15:48:13.593431 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.592593    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerStarted","Data":"3f6527105536dd67c2c97e4036f2dd305110d76045acaf15c0b20a54cbbdd7ec"}
Jul 14 15:48:13.593431 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.592641    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerStarted","Data":"dcd3cd7028eb266391dc4afdfca7f3ac4c2e5c9cfa021c39c931b5098e62fb5b"}
Jul 14 15:48:13.593431 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.592678    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerStarted","Data":"9c0a4b10f4990b596557607b07e1ac22f9062ef57b3658500ccb3caa530404b6"}
Jul 14 15:48:13.593431 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.592714    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerStarted","Data":"1790efb95c182eba4ac32097ae6131ca5a9fb9c04048442ee612a113f16c59e4"}
Jul 14 15:48:13.593431 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.592753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerStarted","Data":"dd80876ce8aa062e98de984b9ace2878f8a49ac13e157be8ee5598f037b42dd7"}
Jul 14 15:48:13.593823 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.593686    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r"
Jul 14 15:48:13.596573 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.596550    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5965d957f-58mld" event={"ID":"812da01e-0d77-451a-8942-eb32e0f0728a","Type":"ContainerStarted","Data":"cd0977c7dacf282054d503ea9f5af7de42b5ddf83874aa2e334437c50c482321"}
Jul 14 15:48:13.596696 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.596610    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5965d957f-58mld" event={"ID":"812da01e-0d77-451a-8942-eb32e0f0728a","Type":"ContainerStarted","Data":"c6ba0bd6801609fd98c61c61ac067a7d5b9341f89a26337a096eaf84f43a13db"}
Jul 14 15:48:13.596696 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.596655    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5965d957f-58mld" event={"ID":"812da01e-0d77-451a-8942-eb32e0f0728a","Type":"ContainerStarted","Data":"ee1241355e01b77cbc0ba080fb21a5b3002418d81a0d1af7a38f15ece33bc4a1"}
Jul 14 15:48:13.601514 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.597153    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:13.603534 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.603506    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2" event={"ID":"ee471a45-f22c-4773-96b7-c96d1da35cee","Type":"ContainerStarted","Data":"96aa2a4d285c398eeb0020e1ab059c4fa92adf5fcb4126aae73b4ea20ccf5256"}
Jul 14 15:48:13.607471 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.607408    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-9f57r" podStartSLOduration=4.43861502 podStartE2EDuration="6.607395395s" totalImagesPullingTime="2.168780375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:13.605194577 +0000 UTC m=+77.124561999" watchObservedRunningTime="2026-07-14 15:48:13.607395395 +0000 UTC m=+77.126762815"
Jul 14 15:48:13.669807 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.669748    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.818696546 podStartE2EDuration="9.66972688s" totalImagesPullingTime="5.826881067s" totalInitContainerRuntime="24.149267ms" isStatefulPod=false podCreationTimestamp="2026-07-14 15:48:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:13.668558034 +0000 UTC m=+77.187925457" watchObservedRunningTime="2026-07-14 15:48:13.66972688 +0000 UTC m=+77.189094625"
Jul 14 15:48:13.696678 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:13.696595    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5965d957f-58mld" podStartSLOduration=1.695863833 podStartE2EDuration="8.696573311s" totalImagesPullingTime="7.000709478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:48:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:13.695361796 +0000 UTC m=+77.214729230" watchObservedRunningTime="2026-07-14 15:48:13.696573311 +0000 UTC m=+77.215940735"
Jul 14 15:48:15.616624 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:15.616593    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerStarted","Data":"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"}
Jul 14 15:48:15.616945 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:15.616706    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerStarted","Data":"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"}
Jul 14 15:48:15.616945 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:15.616735    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerStarted","Data":"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"}
Jul 14 15:48:15.616945 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:15.616761    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerStarted","Data":"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"}
Jul 14 15:48:15.616945 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:15.616792    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerStarted","Data":"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"}
Jul 14 15:48:16.624740 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:16.624701    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerStarted","Data":"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"}
Jul 14 15:48:16.661167 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:16.661109    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2" podStartSLOduration=7.446549242 podStartE2EDuration="9.661089627s" totalImagesPullingTime="2.214540385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:13.733809734 +0000 UTC m=+77.253177181" watchObservedRunningTime="2026-07-14 15:48:16.661089627 +0000 UTC m=+80.180457050"
Jul 14 15:48:16.661697 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:16.661666    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.997156159 podStartE2EDuration="7.661653898s" totalImagesPullingTime="3.640253547s" totalInitContainerRuntime="24.244192ms" isStatefulPod=false podCreationTimestamp="2026-07-14 15:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:48:16.659421816 +0000 UTC m=+80.178789256" watchObservedRunningTime="2026-07-14 15:48:16.661653898 +0000 UTC m=+80.181021318"
Jul 14 15:48:19.615089 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:19.614962    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5965d957f-58mld"
Jul 14 15:48:19.863231 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:19.863188    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:48:21.448706 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:21.448672    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-df9ddd8bf-m2zjx"]
Jul 14 15:48:27.786635 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:27.786591    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:27.787229 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:27.786705    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:35.267462 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:35.267389    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5fb758b489-l2g29" podUID="e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" containerName="console" containerID="cri-o://7bfdf3f1ea8813c098f93e09a34ba53f535767797d2416edfa3bde23f1b1b411" gracePeriod=15
Jul 14 15:48:35.700908 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:35.700872    2560 generic.go:356] "Generic (PLEG): container finished" podID="e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" containerID="7bfdf3f1ea8813c098f93e09a34ba53f535767797d2416edfa3bde23f1b1b411" exitCode=0
Jul 14 15:48:35.701078 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:35.700947    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb758b489-l2g29" event={"ID":"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec","Type":"ContainerDied","Data":"7bfdf3f1ea8813c098f93e09a34ba53f535767797d2416edfa3bde23f1b1b411"}
Jul 14 15:48:36.067110 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.067085    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:48:36.108959 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.108933    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-oauth-serving-cert\") pod \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") "
Jul 14 15:48:36.109125 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.108963    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-oauth-config\") pod \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") "
Jul 14 15:48:36.109125 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.108993    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-config\") pod \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") "
Jul 14 15:48:36.109125 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.109010    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-serving-cert\") pod \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") "
Jul 14 15:48:36.109125 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.109033    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-service-ca\") pod \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") "
Jul 14 15:48:36.109125 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.109072    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrc9f\" (UniqueName: \"kubernetes.io/projected/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-kube-api-access-vrc9f\") pod \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\" (UID: \"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec\") "
Jul 14 15:48:36.109486 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.109428    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-config" (OuterVolumeSpecName: "console-config") pod "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" (UID: "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:48:36.109486 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.109423    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" (UID: "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:48:36.109623 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.109513    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-service-ca" (OuterVolumeSpecName: "service-ca") pod "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" (UID: "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:48:36.111523 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.111480    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" (UID: "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:48:36.111614 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.111543    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-kube-api-access-vrc9f" (OuterVolumeSpecName: "kube-api-access-vrc9f") pod "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" (UID: "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec"). InnerVolumeSpecName "kube-api-access-vrc9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 15:48:36.111614 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.111567    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" (UID: "e74e6c41-9677-4bbf-a1b5-92ffe45df5ec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:48:36.209848 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.209813    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-oauth-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:36.209848 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.209844    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-oauth-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:36.209848 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.209854    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:36.210057 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.209863    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-console-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:36.210057 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.209872    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:36.210057 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.209881    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrc9f\" (UniqueName: \"kubernetes.io/projected/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec-kube-api-access-vrc9f\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:36.705734 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.705705    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb758b489-l2g29" event={"ID":"e74e6c41-9677-4bbf-a1b5-92ffe45df5ec","Type":"ContainerDied","Data":"4caa4cf1904a11db1606c9d0c6cd2a139eeb31cd3317f0681bcb9958a68d564c"}
Jul 14 15:48:36.706113 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.705740    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb758b489-l2g29"
Jul 14 15:48:36.706113 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.705761    2560 scope.go:117] "RemoveContainer" containerID="7bfdf3f1ea8813c098f93e09a34ba53f535767797d2416edfa3bde23f1b1b411"
Jul 14 15:48:36.726811 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.726787    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fb758b489-l2g29"]
Jul 14 15:48:36.732673 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:36.732647    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fb758b489-l2g29"]
Jul 14 15:48:37.031364 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:37.031286    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" path="/var/lib/kubelet/pods/e74e6c41-9677-4bbf-a1b5-92ffe45df5ec/volumes"
Jul 14 15:48:42.584169 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:42.584134    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jxlsn"
Jul 14 15:48:46.469326 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.469269    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-df9ddd8bf-m2zjx" podUID="e0d47717-2e66-4158-b45f-f501ac58f9b1" containerName="console" containerID="cri-o://02e7944033f313906c3260efb25cc0ecae9dc333422adfba0c3a87292751b822" gracePeriod=15
Jul 14 15:48:46.747581 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.747549    2560 generic.go:356] "Generic (PLEG): container finished" podID="e0d47717-2e66-4158-b45f-f501ac58f9b1" containerID="02e7944033f313906c3260efb25cc0ecae9dc333422adfba0c3a87292751b822" exitCode=0
Jul 14 15:48:46.747718 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.747624    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df9ddd8bf-m2zjx" event={"ID":"e0d47717-2e66-4158-b45f-f501ac58f9b1","Type":"ContainerDied","Data":"02e7944033f313906c3260efb25cc0ecae9dc333422adfba0c3a87292751b822"}
Jul 14 15:48:46.747718 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.747670    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df9ddd8bf-m2zjx" event={"ID":"e0d47717-2e66-4158-b45f-f501ac58f9b1","Type":"ContainerDied","Data":"3ef869865583ed89c685858bc06747e6cf0d2578ed52f1a680fb8b168c668015"}
Jul 14 15:48:46.747718 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.747685    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ef869865583ed89c685858bc06747e6cf0d2578ed52f1a680fb8b168c668015"
Jul 14 15:48:46.752834 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.752820    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:46.903496 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.903465    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-oauth-serving-cert\") pod \"e0d47717-2e66-4158-b45f-f501ac58f9b1\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") "
Jul 14 15:48:46.903670 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.903511    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9mvl\" (UniqueName: \"kubernetes.io/projected/e0d47717-2e66-4158-b45f-f501ac58f9b1-kube-api-access-k9mvl\") pod \"e0d47717-2e66-4158-b45f-f501ac58f9b1\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") "
Jul 14 15:48:46.903670 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.903554    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-oauth-config\") pod \"e0d47717-2e66-4158-b45f-f501ac58f9b1\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") "
Jul 14 15:48:46.903670 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.903580    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-config\") pod \"e0d47717-2e66-4158-b45f-f501ac58f9b1\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") "
Jul 14 15:48:46.903670 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.903610    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-trusted-ca-bundle\") pod \"e0d47717-2e66-4158-b45f-f501ac58f9b1\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") "
Jul 14 15:48:46.903670 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.903637    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-serving-cert\") pod \"e0d47717-2e66-4158-b45f-f501ac58f9b1\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") "
Jul 14 15:48:46.903670 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.903652    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-service-ca\") pod \"e0d47717-2e66-4158-b45f-f501ac58f9b1\" (UID: \"e0d47717-2e66-4158-b45f-f501ac58f9b1\") "
Jul 14 15:48:46.904013 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.903890    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e0d47717-2e66-4158-b45f-f501ac58f9b1" (UID: "e0d47717-2e66-4158-b45f-f501ac58f9b1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:48:46.904158 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.904088    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e0d47717-2e66-4158-b45f-f501ac58f9b1" (UID: "e0d47717-2e66-4158-b45f-f501ac58f9b1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:48:46.904212 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.904144    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-config" (OuterVolumeSpecName: "console-config") pod "e0d47717-2e66-4158-b45f-f501ac58f9b1" (UID: "e0d47717-2e66-4158-b45f-f501ac58f9b1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:48:46.904247 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.904220    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-service-ca" (OuterVolumeSpecName: "service-ca") pod "e0d47717-2e66-4158-b45f-f501ac58f9b1" (UID: "e0d47717-2e66-4158-b45f-f501ac58f9b1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:48:46.906092 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.906068    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e0d47717-2e66-4158-b45f-f501ac58f9b1" (UID: "e0d47717-2e66-4158-b45f-f501ac58f9b1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:48:46.906203 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.906155    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d47717-2e66-4158-b45f-f501ac58f9b1-kube-api-access-k9mvl" (OuterVolumeSpecName: "kube-api-access-k9mvl") pod "e0d47717-2e66-4158-b45f-f501ac58f9b1" (UID: "e0d47717-2e66-4158-b45f-f501ac58f9b1"). InnerVolumeSpecName "kube-api-access-k9mvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 15:48:46.906203 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:46.906171    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e0d47717-2e66-4158-b45f-f501ac58f9b1" (UID: "e0d47717-2e66-4158-b45f-f501ac58f9b1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:48:47.005089 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.005006    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-oauth-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:47.005089 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.005036    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9mvl\" (UniqueName: \"kubernetes.io/projected/e0d47717-2e66-4158-b45f-f501ac58f9b1-kube-api-access-k9mvl\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:47.005089 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.005050    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-oauth-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:47.005089 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.005062    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:47.005089 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.005074    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-trusted-ca-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:47.005089 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.005087    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0d47717-2e66-4158-b45f-f501ac58f9b1-console-serving-cert\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:47.005369 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.005098    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d47717-2e66-4158-b45f-f501ac58f9b1-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:48:47.752774 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.752741    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df9ddd8bf-m2zjx"
Jul 14 15:48:47.770273 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.770243    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-df9ddd8bf-m2zjx"]
Jul 14 15:48:47.774628 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.774600    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-df9ddd8bf-m2zjx"]
Jul 14 15:48:47.791667 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.791638    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:47.799262 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:47.799223    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c464d7cf4-4w7b2"
Jul 14 15:48:49.031224 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:48:49.031188    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0d47717-2e66-4158-b45f-f501ac58f9b1" path="/var/lib/kubelet/pods/e0d47717-2e66-4158-b45f-f501ac58f9b1/volumes"
Jul 14 15:49:09.863495 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:09.863466    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:09.884118 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:09.884089    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:10.863432 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:10.863406    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:23.696561 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.696527    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 15:49:23.697052 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.696818    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="alertmanager" containerID="cri-o://dd80876ce8aa062e98de984b9ace2878f8a49ac13e157be8ee5598f037b42dd7" gracePeriod=120
Jul 14 15:49:23.697052 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.696875    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy-metric" containerID="cri-o://3f6527105536dd67c2c97e4036f2dd305110d76045acaf15c0b20a54cbbdd7ec" gracePeriod=120
Jul 14 15:49:23.697052 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.696909    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy-web" containerID="cri-o://9c0a4b10f4990b596557607b07e1ac22f9062ef57b3658500ccb3caa530404b6" gracePeriod=120
Jul 14 15:49:23.697052 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.696931    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="config-reloader" containerID="cri-o://1790efb95c182eba4ac32097ae6131ca5a9fb9c04048442ee612a113f16c59e4" gracePeriod=120
Jul 14 15:49:23.697052 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.696963    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="prom-label-proxy" containerID="cri-o://89a853ba37d788afa5d6d9dd3c906ce0d3d52cca549fc62333cbf9408f481aca" gracePeriod=120
Jul 14 15:49:23.697052 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.696922    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy" containerID="cri-o://dcd3cd7028eb266391dc4afdfca7f3ac4c2e5c9cfa021c39c931b5098e62fb5b" gracePeriod=120
Jul 14 15:49:23.897651 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.897619    2560 generic.go:356] "Generic (PLEG): container finished" podID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerID="89a853ba37d788afa5d6d9dd3c906ce0d3d52cca549fc62333cbf9408f481aca" exitCode=0
Jul 14 15:49:23.897651 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.897645    2560 generic.go:356] "Generic (PLEG): container finished" podID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerID="dcd3cd7028eb266391dc4afdfca7f3ac4c2e5c9cfa021c39c931b5098e62fb5b" exitCode=0
Jul 14 15:49:23.897651 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.897654    2560 generic.go:356] "Generic (PLEG): container finished" podID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerID="1790efb95c182eba4ac32097ae6131ca5a9fb9c04048442ee612a113f16c59e4" exitCode=0
Jul 14 15:49:23.897859 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.897662    2560 generic.go:356] "Generic (PLEG): container finished" podID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerID="dd80876ce8aa062e98de984b9ace2878f8a49ac13e157be8ee5598f037b42dd7" exitCode=0
Jul 14 15:49:23.897859 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.897694    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerDied","Data":"89a853ba37d788afa5d6d9dd3c906ce0d3d52cca549fc62333cbf9408f481aca"}
Jul 14 15:49:23.897859 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.897757    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerDied","Data":"dcd3cd7028eb266391dc4afdfca7f3ac4c2e5c9cfa021c39c931b5098e62fb5b"}
Jul 14 15:49:23.897859 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.897782    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerDied","Data":"1790efb95c182eba4ac32097ae6131ca5a9fb9c04048442ee612a113f16c59e4"}
Jul 14 15:49:23.897859 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:23.897806    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerDied","Data":"dd80876ce8aa062e98de984b9ace2878f8a49ac13e157be8ee5598f037b42dd7"}
Jul 14 15:49:24.906142 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:24.906108    2560 generic.go:356] "Generic (PLEG): container finished" podID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerID="3f6527105536dd67c2c97e4036f2dd305110d76045acaf15c0b20a54cbbdd7ec" exitCode=0
Jul 14 15:49:24.906142 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:24.906137    2560 generic.go:356] "Generic (PLEG): container finished" podID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerID="9c0a4b10f4990b596557607b07e1ac22f9062ef57b3658500ccb3caa530404b6" exitCode=0
Jul 14 15:49:24.906566 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:24.906181    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerDied","Data":"3f6527105536dd67c2c97e4036f2dd305110d76045acaf15c0b20a54cbbdd7ec"}
Jul 14 15:49:24.906566 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:24.906268    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerDied","Data":"9c0a4b10f4990b596557607b07e1ac22f9062ef57b3658500ccb3caa530404b6"}
Jul 14 15:49:24.937996 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:24.937975    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:25.019345 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.019252    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.019345 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.019293    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-web\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.019345 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.019324    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsm6f\" (UniqueName: \"kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-kube-api-access-jsm6f\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.019345 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.019347    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-main-tls\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.019688 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.019375    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-tls-assets\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.019744 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.019407    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-out\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.020667 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.019820    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-cluster-tls-config\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.020667 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.019875    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.020667 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.019927    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-trusted-ca-bundle\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.020667 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.019953    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-web-config\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.020667 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.020115    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-main-db\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.020667 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.020192    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-metrics-client-ca\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.020667 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.020233    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-volume\") pod \"ec7daf54-6c74-4249-88a6-b9ef348267d4\" (UID: \"ec7daf54-6c74-4249-88a6-b9ef348267d4\") "
Jul 14 15:49:25.022656 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.022361    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:25.022656 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.022400    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-out" (OuterVolumeSpecName: "config-out") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 15:49:25.023023 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.023000    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:25.023372 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.023348    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-kube-api-access-jsm6f" (OuterVolumeSpecName: "kube-api-access-jsm6f") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "kube-api-access-jsm6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 15:49:25.023510 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.023481    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 15:49:25.023687 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.023668    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:49:25.023911 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.023877    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-volume" (OuterVolumeSpecName: "config-volume") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:25.023911 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.023884    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:49:25.024114 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.024080    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 15:49:25.024287 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.024264    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:25.024928 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.024729    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:25.028733 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.028696    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:25.035794 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.035771    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-web-config" (OuterVolumeSpecName: "web-config") pod "ec7daf54-6c74-4249-88a6-b9ef348267d4" (UID: "ec7daf54-6c74-4249-88a6-b9ef348267d4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:25.121113 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121083    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-metrics-client-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121113 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121108    2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-volume\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121113 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121119    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121128    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121137    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsm6f\" (UniqueName: \"kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-kube-api-access-jsm6f\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121148    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-main-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121156    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ec7daf54-6c74-4249-88a6-b9ef348267d4-tls-assets\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121163    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-config-out\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121171    2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-cluster-tls-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121179    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121187    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121195    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ec7daf54-6c74-4249-88a6-b9ef348267d4-web-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.121317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.121203    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ec7daf54-6c74-4249-88a6-b9ef348267d4-alertmanager-main-db\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:25.912724 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.912694    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ec7daf54-6c74-4249-88a6-b9ef348267d4","Type":"ContainerDied","Data":"0abc21635cc5f3e74fe420e6668bae3c27438c81e481937b9ca895cfb28eb17a"}
Jul 14 15:49:25.913143 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.912755    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:25.913143 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.912780    2560 scope.go:117] "RemoveContainer" containerID="89a853ba37d788afa5d6d9dd3c906ce0d3d52cca549fc62333cbf9408f481aca"
Jul 14 15:49:25.920952 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.920930    2560 scope.go:117] "RemoveContainer" containerID="3f6527105536dd67c2c97e4036f2dd305110d76045acaf15c0b20a54cbbdd7ec"
Jul 14 15:49:25.928373 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.928355    2560 scope.go:117] "RemoveContainer" containerID="dcd3cd7028eb266391dc4afdfca7f3ac4c2e5c9cfa021c39c931b5098e62fb5b"
Jul 14 15:49:25.935212 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.935190    2560 scope.go:117] "RemoveContainer" containerID="9c0a4b10f4990b596557607b07e1ac22f9062ef57b3658500ccb3caa530404b6"
Jul 14 15:49:25.938346 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.938323    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 15:49:25.941521 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.941500    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 15:49:25.944072 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.944056    2560 scope.go:117] "RemoveContainer" containerID="1790efb95c182eba4ac32097ae6131ca5a9fb9c04048442ee612a113f16c59e4"
Jul 14 15:49:25.951105 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.951085    2560 scope.go:117] "RemoveContainer" containerID="dd80876ce8aa062e98de984b9ace2878f8a49ac13e157be8ee5598f037b42dd7"
Jul 14 15:49:25.959918 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.959894    2560 scope.go:117] "RemoveContainer" containerID="025f9ab81814d14278fd4cd98c51a0a24ac6d83cef13243b00aa593fa8340b13"
Jul 14 15:49:25.971961 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.971940    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 15:49:25.972240 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972227    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy-web"
Jul 14 15:49:25.972240 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972241    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy-web"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972252    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="prom-label-proxy"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972258    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="prom-label-proxy"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972266    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="config-reloader"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972271    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="config-reloader"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972281    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972286    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972294    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy-metric"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972299    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy-metric"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972308    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="alertmanager"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972313    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="alertmanager"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972321    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" containerName="console"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972325    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" containerName="console"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972331    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="init-config-reloader"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972336    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="init-config-reloader"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972345    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d47717-2e66-4158-b45f-f501ac58f9b1" containerName="console"
Jul 14 15:49:25.972358 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972350    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d47717-2e66-4158-b45f-f501ac58f9b1" containerName="console"
Jul 14 15:49:25.972923 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972397    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy-metric"
Jul 14 15:49:25.972923 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972405    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="prom-label-proxy"
Jul 14 15:49:25.972923 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972411    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e74e6c41-9677-4bbf-a1b5-92ffe45df5ec" containerName="console"
Jul 14 15:49:25.972923 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972416    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="config-reloader"
Jul 14 15:49:25.972923 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972422    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="alertmanager"
Jul 14 15:49:25.972923 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972428    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy"
Jul 14 15:49:25.972923 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972433    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d47717-2e66-4158-b45f-f501ac58f9b1" containerName="console"
Jul 14 15:49:25.972923 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.972438    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" containerName="kube-rbac-proxy-web"
Jul 14 15:49:25.976082 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.976066    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:25.979223 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.979195    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 14 15:49:25.979589 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.979569    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 14 15:49:25.980166 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.979841    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 14 15:49:25.980166 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.980011    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 14 15:49:25.980317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.980300    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-njrlw\""
Jul 14 15:49:25.980400 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.980378    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 14 15:49:25.980720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.980598    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 14 15:49:25.980720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.980630    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 14 15:49:25.981039 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.981016    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 14 15:49:25.984568 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.984547    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 14 15:49:26.000325 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:25.996935    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 15:49:26.028311 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028239    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028311 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028277    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-web-config\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028311 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028296    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93a75ec2-b796-4898-876e-4a08aef3264f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028592 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028317    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028592 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028355    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028592 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028372    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028592 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028392    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93a75ec2-b796-4898-876e-4a08aef3264f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028592 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028438    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksspc\" (UniqueName: \"kubernetes.io/projected/93a75ec2-b796-4898-876e-4a08aef3264f-kube-api-access-ksspc\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028592 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028525    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/93a75ec2-b796-4898-876e-4a08aef3264f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028592 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028557    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-config-volume\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028592 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028586    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/93a75ec2-b796-4898-876e-4a08aef3264f-config-out\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028862 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028613    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.028862 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.028640    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/93a75ec2-b796-4898-876e-4a08aef3264f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.129619 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129565    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.129619 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129634    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-web-config\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.129885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129653    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93a75ec2-b796-4898-876e-4a08aef3264f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.129885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129677    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.129885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129695    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.129885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129713    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.129885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129742    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93a75ec2-b796-4898-876e-4a08aef3264f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.129885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129764    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksspc\" (UniqueName: \"kubernetes.io/projected/93a75ec2-b796-4898-876e-4a08aef3264f-kube-api-access-ksspc\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.130156 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129892    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/93a75ec2-b796-4898-876e-4a08aef3264f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.130156 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129926    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-config-volume\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.130156 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129960    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/93a75ec2-b796-4898-876e-4a08aef3264f-config-out\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.130156 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.129989    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.130156 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.130015    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/93a75ec2-b796-4898-876e-4a08aef3264f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.130542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.130504    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/93a75ec2-b796-4898-876e-4a08aef3264f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.130653 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.130523    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93a75ec2-b796-4898-876e-4a08aef3264f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.131061 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.130927    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93a75ec2-b796-4898-876e-4a08aef3264f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.133054 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.133028    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.133320 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.133298    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.133442 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.133421    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/93a75ec2-b796-4898-876e-4a08aef3264f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.133805 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.133767    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.133921 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.133871    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-web-config\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.133986 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.133937    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-config-volume\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.134106 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.134086    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.134441 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.134418    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/93a75ec2-b796-4898-876e-4a08aef3264f-config-out\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.134665 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.134647    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/93a75ec2-b796-4898-876e-4a08aef3264f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.139885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.139865    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksspc\" (UniqueName: \"kubernetes.io/projected/93a75ec2-b796-4898-876e-4a08aef3264f-kube-api-access-ksspc\") pod \"alertmanager-main-0\" (UID: \"93a75ec2-b796-4898-876e-4a08aef3264f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.287904 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.287826    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 15:49:26.431461 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.431416    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 15:49:26.434470 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:49:26.434428    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93a75ec2_b796_4898_876e_4a08aef3264f.slice/crio-5a08898cd3ab22d459664c413e3cb8991b62ce32ec383d122c31fc721c5935c6 WatchSource:0}: Error finding container 5a08898cd3ab22d459664c413e3cb8991b62ce32ec383d122c31fc721c5935c6: Status 404 returned error can't find the container with id 5a08898cd3ab22d459664c413e3cb8991b62ce32ec383d122c31fc721c5935c6
Jul 14 15:49:26.917978 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.917943    2560 generic.go:356] "Generic (PLEG): container finished" podID="93a75ec2-b796-4898-876e-4a08aef3264f" containerID="68cb492312d837944617d9ea940a4df6827df1591768bde097b5b07996225594" exitCode=0
Jul 14 15:49:26.918425 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.918037    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a75ec2-b796-4898-876e-4a08aef3264f","Type":"ContainerDied","Data":"68cb492312d837944617d9ea940a4df6827df1591768bde097b5b07996225594"}
Jul 14 15:49:26.918425 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:26.918113    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a75ec2-b796-4898-876e-4a08aef3264f","Type":"ContainerStarted","Data":"5a08898cd3ab22d459664c413e3cb8991b62ce32ec383d122c31fc721c5935c6"}
Jul 14 15:49:27.032899 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.032871    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec7daf54-6c74-4249-88a6-b9ef348267d4" path="/var/lib/kubelet/pods/ec7daf54-6c74-4249-88a6-b9ef348267d4/volumes"
Jul 14 15:49:27.673947 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.673910    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"]
Jul 14 15:49:27.680687 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.680660    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.683596 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.683571    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-snxs6\""
Jul 14 15:49:27.685337 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.683952    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 14 15:49:27.685841 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.684005    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 14 15:49:27.685841 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.684104    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 14 15:49:27.685841 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.684169    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 14 15:49:27.686394 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.686375    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 14 15:49:27.690726 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.690707    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 14 15:49:27.692014 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.691950    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"]
Jul 14 15:49:27.746826 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.746791    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cc47fc7-d572-4281-9e26-71c1ef812bea-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.746826 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.746831    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cc47fc7-d572-4281-9e26-71c1ef812bea-serving-certs-ca-bundle\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.747037 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.746862    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-secret-telemeter-client\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.747037 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.746886    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-federate-client-tls\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.747037 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.746950    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-telemeter-client-tls\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.747037 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.747011    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.747037 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.747030    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1cc47fc7-d572-4281-9e26-71c1ef812bea-metrics-client-ca\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.747218 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.747054    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9xl7\" (UniqueName: \"kubernetes.io/projected/1cc47fc7-d572-4281-9e26-71c1ef812bea-kube-api-access-c9xl7\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.848491 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.848425    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.848639 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.848495    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1cc47fc7-d572-4281-9e26-71c1ef812bea-metrics-client-ca\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.848639 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.848540    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9xl7\" (UniqueName: \"kubernetes.io/projected/1cc47fc7-d572-4281-9e26-71c1ef812bea-kube-api-access-c9xl7\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.848639 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.848604    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cc47fc7-d572-4281-9e26-71c1ef812bea-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.848639 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.848632    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cc47fc7-d572-4281-9e26-71c1ef812bea-serving-certs-ca-bundle\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.848857 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.848663    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-secret-telemeter-client\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.848857 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.848718    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-federate-client-tls\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.848857 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.848782    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-telemeter-client-tls\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.850059 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.850028    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cc47fc7-d572-4281-9e26-71c1ef812bea-serving-certs-ca-bundle\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.851082 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.851053    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cc47fc7-d572-4281-9e26-71c1ef812bea-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.851234 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.851086    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1cc47fc7-d572-4281-9e26-71c1ef812bea-metrics-client-ca\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.851661 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.851639    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-federate-client-tls\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.851732 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.851715    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-secret-telemeter-client\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.851769 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.851756    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.851802 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.851780    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1cc47fc7-d572-4281-9e26-71c1ef812bea-telemeter-client-tls\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.857968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.857943    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9xl7\" (UniqueName: \"kubernetes.io/projected/1cc47fc7-d572-4281-9e26-71c1ef812bea-kube-api-access-c9xl7\") pod \"telemeter-client-6b54b766fc-c5k2m\" (UID: \"1cc47fc7-d572-4281-9e26-71c1ef812bea\") " pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:27.926835 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.926750    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a75ec2-b796-4898-876e-4a08aef3264f","Type":"ContainerStarted","Data":"6b7c94361fbafed169c41aac9bb500c81a549fb53920caf0abeb418f451a4c62"}
Jul 14 15:49:27.926835 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.926824    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a75ec2-b796-4898-876e-4a08aef3264f","Type":"ContainerStarted","Data":"8947b5b1dc2129c275b5e81af8e7b685d82f526050eb1e767a00dbf6dcf2ebe2"}
Jul 14 15:49:27.927228 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.926850    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a75ec2-b796-4898-876e-4a08aef3264f","Type":"ContainerStarted","Data":"1e3dd6d46262fc011047c81b621475dc71880e7032fec6a18d2a4cebf1fe5b42"}
Jul 14 15:49:27.927228 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.926877    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a75ec2-b796-4898-876e-4a08aef3264f","Type":"ContainerStarted","Data":"988ec5c513d29656fa72484779d3671a38430fccd76ad8db707095e3079864f3"}
Jul 14 15:49:27.927228 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.926905    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a75ec2-b796-4898-876e-4a08aef3264f","Type":"ContainerStarted","Data":"a23af00de279b2a0b90dd4384d43395c6598cb88f104fc9bdecf75235db33c07"}
Jul 14 15:49:27.927228 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.926934    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93a75ec2-b796-4898-876e-4a08aef3264f","Type":"ContainerStarted","Data":"2f2608361d441dd2f4ec59bc8d6f9b3ceb258cc2f50aeeffee231f5b5fbc21af"}
Jul 14 15:49:27.957356 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.957286    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.942637928 podStartE2EDuration="2.957265006s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.627078ms" isStatefulPod=false podCreationTimestamp="2026-07-14 15:49:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:49:27.955670472 +0000 UTC m=+151.475037893" watchObservedRunningTime="2026-07-14 15:49:27.957265006 +0000 UTC m=+151.476632428"
Jul 14 15:49:27.982347 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.982310    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 15:49:27.982749 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.982713    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="prometheus" containerID="cri-o://cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21" gracePeriod=600
Jul 14 15:49:27.983114 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.983081    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy-thanos" containerID="cri-o://add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb" gracePeriod=600
Jul 14 15:49:27.983246 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.983219    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy" containerID="cri-o://7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9" gracePeriod=600
Jul 14 15:49:27.983381 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.983327    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy-web" containerID="cri-o://722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b" gracePeriod=600
Jul 14 15:49:27.983440 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.983409    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="thanos-sidecar" containerID="cri-o://15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5" gracePeriod=600
Jul 14 15:49:27.983520 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.983508    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="config-reloader" containerID="cri-o://fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226" gracePeriod=600
Jul 14 15:49:27.997931 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:27.997854    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"
Jul 14 15:49:28.229290 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.229265    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:28.252920 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.252890    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b54b766fc-c5k2m"]
Jul 14 15:49:28.254894 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:49:28.254863    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc47fc7_d572_4281_9e26_71c1ef812bea.slice/crio-8f45bbf3c8218c0fa9a50e2acb3e830052753ff3c2c6a095c5f0aa6b943916ee WatchSource:0}: Error finding container 8f45bbf3c8218c0fa9a50e2acb3e830052753ff3c2c6a095c5f0aa6b943916ee: Status 404 returned error can't find the container with id 8f45bbf3c8218c0fa9a50e2acb3e830052753ff3c2c6a095c5f0aa6b943916ee
Jul 14 15:49:28.354910 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.354875    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.354910 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.354913    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-1\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355121 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.354930    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-0\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355121 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.354947    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-db\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355121 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355068    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-trusted-ca-bundle\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355121 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355118    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf7g2\" (UniqueName: \"kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-kube-api-access-xf7g2\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355350 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355147    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-grpc-tls\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355350 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355172    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-tls-assets\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355350 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355201    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-kubelet-serving-ca-bundle\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355350 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355247    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-metrics-client-certs\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355350 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355249    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:49:28.355350 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355283    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-kube-rbac-proxy\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355350 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355312    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-web-config\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355350 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355352    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-thanos-prometheus-http-client-file\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355391    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355406    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:49:28.355968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355429    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-serving-certs-ca-bundle\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355480    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-config-out\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355515    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-tls\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355573    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-2\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355604    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-config\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355643    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-metrics-client-ca\") pod \"38c5d7de-be40-47f8-ba10-02f952515179\" (UID: \"38c5d7de-be40-47f8-ba10-02f952515179\") "
Jul 14 15:49:28.355968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355943    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.355968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355951    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:49:28.355968 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.355961    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.356547 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.356110    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:49:28.356547 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.356192    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 15:49:28.356547 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.356270    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:49:28.356701 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.356623    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:49:28.357976 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.357563    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 15:49:28.358167 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.358136    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:28.358832 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.358793    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:28.359316 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.359272    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-config" (OuterVolumeSpecName: "config") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:28.360044 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.360018    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-config-out" (OuterVolumeSpecName: "config-out") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 15:49:28.360423 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.360386    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 15:49:28.360562 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.360535    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:28.360626 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.360603    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:28.360893 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.360857    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-kube-api-access-xf7g2" (OuterVolumeSpecName: "kube-api-access-xf7g2") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "kube-api-access-xf7g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 15:49:28.362095 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.362061    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:28.362690 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.362661    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:28.363465 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.363419    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:28.372508 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.372487    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-web-config" (OuterVolumeSpecName: "web-config") pod "38c5d7de-be40-47f8-ba10-02f952515179" (UID: "38c5d7de-be40-47f8-ba10-02f952515179"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 15:49:28.457283 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457249    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-web-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457283 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457277    2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457283 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457290    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457301    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457310    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-config-out\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457319    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457327    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457335    2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-config\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457344    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-metrics-client-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457353    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457361    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457369    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38c5d7de-be40-47f8-ba10-02f952515179-prometheus-k8s-db\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457376    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf7g2\" (UniqueName: \"kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-kube-api-access-xf7g2\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457384    2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-grpc-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457392    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38c5d7de-be40-47f8-ba10-02f952515179-tls-assets\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457401    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38c5d7de-be40-47f8-ba10-02f952515179-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457409    2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-metrics-client-certs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.457542 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.457417    2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38c5d7de-be40-47f8-ba10-02f952515179-secret-kube-rbac-proxy\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 15:49:28.932995 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.932944    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m" event={"ID":"1cc47fc7-d572-4281-9e26-71c1ef812bea","Type":"ContainerStarted","Data":"8f45bbf3c8218c0fa9a50e2acb3e830052753ff3c2c6a095c5f0aa6b943916ee"}
Jul 14 15:49:28.936511 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936479    2560 generic.go:356] "Generic (PLEG): container finished" podID="38c5d7de-be40-47f8-ba10-02f952515179" containerID="add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb" exitCode=0
Jul 14 15:49:28.936511 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936512    2560 generic.go:356] "Generic (PLEG): container finished" podID="38c5d7de-be40-47f8-ba10-02f952515179" containerID="7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9" exitCode=0
Jul 14 15:49:28.936704 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936524    2560 generic.go:356] "Generic (PLEG): container finished" podID="38c5d7de-be40-47f8-ba10-02f952515179" containerID="722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b" exitCode=0
Jul 14 15:49:28.936704 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936533    2560 generic.go:356] "Generic (PLEG): container finished" podID="38c5d7de-be40-47f8-ba10-02f952515179" containerID="15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5" exitCode=0
Jul 14 15:49:28.936704 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936541    2560 generic.go:356] "Generic (PLEG): container finished" podID="38c5d7de-be40-47f8-ba10-02f952515179" containerID="fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226" exitCode=0
Jul 14 15:49:28.936704 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936549    2560 generic.go:356] "Generic (PLEG): container finished" podID="38c5d7de-be40-47f8-ba10-02f952515179" containerID="cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21" exitCode=0
Jul 14 15:49:28.936704 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936568    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:28.936704 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936567    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerDied","Data":"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"}
Jul 14 15:49:28.936704 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936653    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerDied","Data":"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"}
Jul 14 15:49:28.936704 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936700    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerDied","Data":"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"}
Jul 14 15:49:28.937070 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936713    2560 scope.go:117] "RemoveContainer" containerID="add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"
Jul 14 15:49:28.937070 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936742    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerDied","Data":"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"}
Jul 14 15:49:28.937070 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936783    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerDied","Data":"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"}
Jul 14 15:49:28.937070 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936826    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerDied","Data":"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"}
Jul 14 15:49:28.937070 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.936868    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38c5d7de-be40-47f8-ba10-02f952515179","Type":"ContainerDied","Data":"5d9dfd54d4b6947f3be94f8714b5e5e4f98197cdf14d8985dedd854f400d2923"}
Jul 14 15:49:28.946792 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.946768    2560 scope.go:117] "RemoveContainer" containerID="7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"
Jul 14 15:49:28.954477 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.954461    2560 scope.go:117] "RemoveContainer" containerID="722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"
Jul 14 15:49:28.961939 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.961922    2560 scope.go:117] "RemoveContainer" containerID="15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"
Jul 14 15:49:28.962347 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.962326    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 15:49:28.966763 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.966743    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 15:49:28.969295 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.969281    2560 scope.go:117] "RemoveContainer" containerID="fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"
Jul 14 15:49:28.975770 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.975756    2560 scope.go:117] "RemoveContainer" containerID="cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"
Jul 14 15:49:28.982798 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.982783    2560 scope.go:117] "RemoveContainer" containerID="602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"
Jul 14 15:49:28.989632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.989514    2560 scope.go:117] "RemoveContainer" containerID="add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"
Jul 14 15:49:28.989787 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:49:28.989768    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": container with ID starting with add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb not found: ID does not exist" containerID="add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"
Jul 14 15:49:28.989831 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.989795    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"} err="failed to get container status \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": rpc error: code = NotFound desc = could not find container \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": container with ID starting with add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb not found: ID does not exist"
Jul 14 15:49:28.989831 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.989829    2560 scope.go:117] "RemoveContainer" containerID="7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"
Jul 14 15:49:28.990080 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:49:28.990061    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": container with ID starting with 7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9 not found: ID does not exist" containerID="7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"
Jul 14 15:49:28.990119 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.990087    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"} err="failed to get container status \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": rpc error: code = NotFound desc = could not find container \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": container with ID starting with 7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9 not found: ID does not exist"
Jul 14 15:49:28.990119 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.990105    2560 scope.go:117] "RemoveContainer" containerID="722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"
Jul 14 15:49:28.990340 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:49:28.990324    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": container with ID starting with 722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b not found: ID does not exist" containerID="722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"
Jul 14 15:49:28.990396 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.990343    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"} err="failed to get container status \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": rpc error: code = NotFound desc = could not find container \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": container with ID starting with 722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b not found: ID does not exist"
Jul 14 15:49:28.990396 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.990357    2560 scope.go:117] "RemoveContainer" containerID="15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"
Jul 14 15:49:28.990639 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:49:28.990624    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": container with ID starting with 15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5 not found: ID does not exist" containerID="15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"
Jul 14 15:49:28.990679 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.990646    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"} err="failed to get container status \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": rpc error: code = NotFound desc = could not find container \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": container with ID starting with 15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5 not found: ID does not exist"
Jul 14 15:49:28.990679 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.990663    2560 scope.go:117] "RemoveContainer" containerID="fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"
Jul 14 15:49:28.990896 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:49:28.990879    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": container with ID starting with fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226 not found: ID does not exist" containerID="fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"
Jul 14 15:49:28.990940 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.990899    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"} err="failed to get container status \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": rpc error: code = NotFound desc = could not find container \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": container with ID starting with fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226 not found: ID does not exist"
Jul 14 15:49:28.990940 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.990913    2560 scope.go:117] "RemoveContainer" containerID="cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"
Jul 14 15:49:28.991144 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:49:28.991127    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": container with ID starting with cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21 not found: ID does not exist" containerID="cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"
Jul 14 15:49:28.991183 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.991151    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"} err="failed to get container status \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": rpc error: code = NotFound desc = could not find container \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": container with ID starting with cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21 not found: ID does not exist"
Jul 14 15:49:28.991183 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.991164    2560 scope.go:117] "RemoveContainer" containerID="602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"
Jul 14 15:49:28.991360 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:49:28.991345    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": container with ID starting with 602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64 not found: ID does not exist" containerID="602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"
Jul 14 15:49:28.991401 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.991370    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"} err="failed to get container status \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": rpc error: code = NotFound desc = could not find container \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": container with ID starting with 602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64 not found: ID does not exist"
Jul 14 15:49:28.991401 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.991381    2560 scope.go:117] "RemoveContainer" containerID="add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"
Jul 14 15:49:28.991815 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.991777    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"} err="failed to get container status \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": rpc error: code = NotFound desc = could not find container \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": container with ID starting with add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb not found: ID does not exist"
Jul 14 15:49:28.991909 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.991818    2560 scope.go:117] "RemoveContainer" containerID="7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"
Jul 14 15:49:28.992084 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.992062    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"} err="failed to get container status \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": rpc error: code = NotFound desc = could not find container \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": container with ID starting with 7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9 not found: ID does not exist"
Jul 14 15:49:28.992154 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.992085    2560 scope.go:117] "RemoveContainer" containerID="722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"
Jul 14 15:49:28.992326 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.992309    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"} err="failed to get container status \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": rpc error: code = NotFound desc = could not find container \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": container with ID starting with 722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b not found: ID does not exist"
Jul 14 15:49:28.992326 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.992325    2560 scope.go:117] "RemoveContainer" containerID="15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"
Jul 14 15:49:28.993059 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.992897    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"} err="failed to get container status \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": rpc error: code = NotFound desc = could not find container \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": container with ID starting with 15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5 not found: ID does not exist"
Jul 14 15:49:28.993059 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.992922    2560 scope.go:117] "RemoveContainer" containerID="fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.993255    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"} err="failed to get container status \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": rpc error: code = NotFound desc = could not find container \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": container with ID starting with fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226 not found: ID does not exist"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.993280    2560 scope.go:117] "RemoveContainer" containerID="cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.993609    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"} err="failed to get container status \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": rpc error: code = NotFound desc = could not find container \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": container with ID starting with cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21 not found: ID does not exist"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.993631    2560 scope.go:117] "RemoveContainer" containerID="602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.993970    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"} err="failed to get container status \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": rpc error: code = NotFound desc = could not find container \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": container with ID starting with 602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64 not found: ID does not exist"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.993993    2560 scope.go:117] "RemoveContainer" containerID="add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.994310    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"} err="failed to get container status \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": rpc error: code = NotFound desc = could not find container \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": container with ID starting with add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb not found: ID does not exist"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.994330    2560 scope.go:117] "RemoveContainer" containerID="7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.994755    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"} err="failed to get container status \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": rpc error: code = NotFound desc = could not find container \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": container with ID starting with 7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9 not found: ID does not exist"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.994779    2560 scope.go:117] "RemoveContainer" containerID="722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.995065    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"} err="failed to get container status \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": rpc error: code = NotFound desc = could not find container \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": container with ID starting with 722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b not found: ID does not exist"
Jul 14 15:49:28.996720 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.995086    2560 scope.go:117] "RemoveContainer" containerID="15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"
Jul 14 15:49:28.997282 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.996764    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"} err="failed to get container status \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": rpc error: code = NotFound desc = could not find container \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": container with ID starting with 15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5 not found: ID does not exist"
Jul 14 15:49:28.997282 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.996781    2560 scope.go:117] "RemoveContainer" containerID="fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"
Jul 14 15:49:28.997282 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997139    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"} err="failed to get container status \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": rpc error: code = NotFound desc = could not find container \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": container with ID starting with fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226 not found: ID does not exist"
Jul 14 15:49:28.997282 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997169    2560 scope.go:117] "RemoveContainer" containerID="cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"
Jul 14 15:49:28.997282 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997250    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 15:49:28.997703 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997683    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="thanos-sidecar"
Jul 14 15:49:28.997703 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997707    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="thanos-sidecar"
Jul 14 15:49:28.997870 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997700    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"} err="failed to get container status \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": rpc error: code = NotFound desc = could not find container \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": container with ID starting with cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21 not found: ID does not exist"
Jul 14 15:49:28.997870 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997734    2560 scope.go:117] "RemoveContainer" containerID="602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"
Jul 14 15:49:28.997870 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997720    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="prometheus"
Jul 14 15:49:28.997870 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997796    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="prometheus"
Jul 14 15:49:28.997870 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997816    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy"
Jul 14 15:49:28.997870 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997825    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy"
Jul 14 15:49:28.997870 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997850    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="init-config-reloader"
Jul 14 15:49:28.997870 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997859    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="init-config-reloader"
Jul 14 15:49:28.997870 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997869    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy-thanos"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997877    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy-thanos"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997901    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy-web"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997910    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy-web"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997925    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="config-reloader"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.997933    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="config-reloader"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998006    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"} err="failed to get container status \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": rpc error: code = NotFound desc = could not find container \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": container with ID starting with 602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64 not found: ID does not exist"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998027    2560 scope.go:117] "RemoveContainer" containerID="add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998028    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy-thanos"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998083    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="config-reloader"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998110    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998121    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="thanos-sidecar"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998132    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="prometheus"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998141    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38c5d7de-be40-47f8-ba10-02f952515179" containerName="kube-rbac-proxy-web"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998258    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"} err="failed to get container status \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": rpc error: code = NotFound desc = could not find container \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": container with ID starting with add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb not found: ID does not exist"
Jul 14 15:49:28.998343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998279    2560 scope.go:117] "RemoveContainer" containerID="7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"
Jul 14 15:49:28.999053 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998503    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"} err="failed to get container status \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": rpc error: code = NotFound desc = could not find container \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": container with ID starting with 7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9 not found: ID does not exist"
Jul 14 15:49:28.999053 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998526    2560 scope.go:117] "RemoveContainer" containerID="722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"
Jul 14 15:49:28.999053 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998823    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"} err="failed to get container status \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": rpc error: code = NotFound desc = could not find container \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": container with ID starting with 722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b not found: ID does not exist"
Jul 14 15:49:28.999053 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.998849    2560 scope.go:117] "RemoveContainer" containerID="15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"
Jul 14 15:49:28.999244 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.999070    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"} err="failed to get container status \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": rpc error: code = NotFound desc = could not find container \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": container with ID starting with 15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5 not found: ID does not exist"
Jul 14 15:49:28.999244 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.999087    2560 scope.go:117] "RemoveContainer" containerID="fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"
Jul 14 15:49:28.999347 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.999290    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"} err="failed to get container status \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": rpc error: code = NotFound desc = could not find container \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": container with ID starting with fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226 not found: ID does not exist"
Jul 14 15:49:28.999347 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.999322    2560 scope.go:117] "RemoveContainer" containerID="cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"
Jul 14 15:49:28.999864 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.999833    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"} err="failed to get container status \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": rpc error: code = NotFound desc = could not find container \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": container with ID starting with cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21 not found: ID does not exist"
Jul 14 15:49:28.999864 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:28.999861    2560 scope.go:117] "RemoveContainer" containerID="602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"
Jul 14 15:49:29.000206 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.000166    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"} err="failed to get container status \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": rpc error: code = NotFound desc = could not find container \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": container with ID starting with 602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64 not found: ID does not exist"
Jul 14 15:49:29.000206 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.000183    2560 scope.go:117] "RemoveContainer" containerID="add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"
Jul 14 15:49:29.000425 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.000377    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"} err="failed to get container status \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": rpc error: code = NotFound desc = could not find container \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": container with ID starting with add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb not found: ID does not exist"
Jul 14 15:49:29.000425 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.000392    2560 scope.go:117] "RemoveContainer" containerID="7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"
Jul 14 15:49:29.000673 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.000612    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"} err="failed to get container status \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": rpc error: code = NotFound desc = could not find container \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": container with ID starting with 7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9 not found: ID does not exist"
Jul 14 15:49:29.000673 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.000640    2560 scope.go:117] "RemoveContainer" containerID="722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"
Jul 14 15:49:29.002801 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.002663    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"} err="failed to get container status \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": rpc error: code = NotFound desc = could not find container \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": container with ID starting with 722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b not found: ID does not exist"
Jul 14 15:49:29.002801 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.002705    2560 scope.go:117] "RemoveContainer" containerID="15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"
Jul 14 15:49:29.003676 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.003561    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"} err="failed to get container status \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": rpc error: code = NotFound desc = could not find container \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": container with ID starting with 15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5 not found: ID does not exist"
Jul 14 15:49:29.003676 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.003587    2560 scope.go:117] "RemoveContainer" containerID="fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"
Jul 14 15:49:29.003898 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.003863    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"} err="failed to get container status \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": rpc error: code = NotFound desc = could not find container \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": container with ID starting with fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226 not found: ID does not exist"
Jul 14 15:49:29.003898 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.003887    2560 scope.go:117] "RemoveContainer" containerID="cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"
Jul 14 15:49:29.004164 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.004136    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"} err="failed to get container status \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": rpc error: code = NotFound desc = could not find container \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": container with ID starting with cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21 not found: ID does not exist"
Jul 14 15:49:29.004388 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.004166    2560 scope.go:117] "RemoveContainer" containerID="602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"
Jul 14 15:49:29.004479 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.004394    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"} err="failed to get container status \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": rpc error: code = NotFound desc = could not find container \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": container with ID starting with 602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64 not found: ID does not exist"
Jul 14 15:49:29.004479 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.004413    2560 scope.go:117] "RemoveContainer" containerID="add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"
Jul 14 15:49:29.004680 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.004660    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb"} err="failed to get container status \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": rpc error: code = NotFound desc = could not find container \"add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb\": container with ID starting with add780c98eea54fe029e3bd1ed52206abf5af65868e0e980e3a0ccfdce91dceb not found: ID does not exist"
Jul 14 15:49:29.004755 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.004683    2560 scope.go:117] "RemoveContainer" containerID="7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"
Jul 14 15:49:29.004962 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.004939    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9"} err="failed to get container status \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": rpc error: code = NotFound desc = could not find container \"7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9\": container with ID starting with 7607b683e699bc9480241987c1d29bd45f709c4d9183f8c2cba7831f6cbbb1e9 not found: ID does not exist"
Jul 14 15:49:29.005031 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.004963    2560 scope.go:117] "RemoveContainer" containerID="722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"
Jul 14 15:49:29.005075 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.005058    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.005791 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.005406    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b"} err="failed to get container status \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": rpc error: code = NotFound desc = could not find container \"722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b\": container with ID starting with 722ab0ea05db9bd873aa8032e1e46b1f95272c3bff600e6100015940a6ebae1b not found: ID does not exist"
Jul 14 15:49:29.005791 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.005432    2560 scope.go:117] "RemoveContainer" containerID="15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"
Jul 14 15:49:29.005791 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.005735    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5"} err="failed to get container status \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": rpc error: code = NotFound desc = could not find container \"15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5\": container with ID starting with 15af2d7f3f4aacee9353021f35c7a8eb77f556d71327d9bd2a68ca8a040b52a5 not found: ID does not exist"
Jul 14 15:49:29.005791 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.005759    2560 scope.go:117] "RemoveContainer" containerID="fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"
Jul 14 15:49:29.008686 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.008024    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 14 15:49:29.008686 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.008103    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mhrsz\""
Jul 14 15:49:29.008686 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.008291    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 14 15:49:29.008686 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.008406    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 14 15:49:29.008686 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.008292    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 14 15:49:29.009413 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.009211    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226"} err="failed to get container status \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": rpc error: code = NotFound desc = could not find container \"fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226\": container with ID starting with fc0f1dc529c23ec7c26fe8f62cf53808d4d24b96a1685afad455ea4012e4d226 not found: ID does not exist"
Jul 14 15:49:29.009413 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.009241    2560 scope.go:117] "RemoveContainer" containerID="cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"
Jul 14 15:49:29.009634 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.009616    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 14 15:49:29.012507 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.009889    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21"} err="failed to get container status \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": rpc error: code = NotFound desc = could not find container \"cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21\": container with ID starting with cb00ce8bb102b48a09a889e1c2a60f3907f789f3a6722e1509464267e2d59b21 not found: ID does not exist"
Jul 14 15:49:29.012507 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.009911    2560 scope.go:117] "RemoveContainer" containerID="602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"
Jul 14 15:49:29.012507 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.009920    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 14 15:49:29.012507 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.011196    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 14 15:49:29.012507 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.011436    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 14 15:49:29.012507 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.011485    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 14 15:49:29.012507 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.011675    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 14 15:49:29.012507 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.011898    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5b6sg8bv5721m\""
Jul 14 15:49:29.012507 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.012157    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 14 15:49:29.012507 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.012291    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 14 15:49:29.013371 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.013335    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64"} err="failed to get container status \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": rpc error: code = NotFound desc = could not find container \"602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64\": container with ID starting with 602555d22afcc1fe89a6eb12ede1da0ccdb026eace5c9febe8243e69ef2fea64 not found: ID does not exist"
Jul 14 15:49:29.014408 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.014383    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 14 15:49:29.016749 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.016729    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 14 15:49:29.033039 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.032985    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38c5d7de-be40-47f8-ba10-02f952515179" path="/var/lib/kubelet/pods/38c5d7de-be40-47f8-ba10-02f952515179/volumes"
Jul 14 15:49:29.062670 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.062643    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.062787 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.062772    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-config\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.062865 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.062797    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.062865 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.062822    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.062865 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.062846    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063010 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.062865    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92387b00-938f-451a-8aac-776a13df9d61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063010 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.062887    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063010 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.062910    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063010 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.062938    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063214 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.063027    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063214 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.063145    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063310 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.063245    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063310 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.063270    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crfjk\" (UniqueName: \"kubernetes.io/projected/92387b00-938f-451a-8aac-776a13df9d61-kube-api-access-crfjk\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063310 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.063293    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92387b00-938f-451a-8aac-776a13df9d61-config-out\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.063310    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.063330    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-web-config\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.063438    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.063482    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063556 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.063540    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.063800 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.063565    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.088291 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.088264    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 15:49:29.164832 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.164800    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.164978 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.164852    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-config\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.164978 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.164869    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.164978 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.164884    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.164978 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.164899    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.164978 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.164915    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92387b00-938f-451a-8aac-776a13df9d61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.164978 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.164934    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.164978 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.164956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.165320 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.164994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.165320 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.165043    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.165861    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.165974    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166006    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166032    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crfjk\" (UniqueName: \"kubernetes.io/projected/92387b00-938f-451a-8aac-776a13df9d61-kube-api-access-crfjk\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166055    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92387b00-938f-451a-8aac-776a13df9d61-config-out\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166086    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166112    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-web-config\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166165    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166188    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166225    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166251    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166285    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.166632 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166288    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.167332 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166950    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.167332 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.166961    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.167332 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.167324    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.167982 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.167954    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.168335 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.168308    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.169832 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.169791    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.170150 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.170122    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.170227 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.170137    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-config\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.170346 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.170324    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.170493 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.170469    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.170704 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.170680    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-web-config\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.170941 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.170922    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.171293 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.171265    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/92387b00-938f-451a-8aac-776a13df9d61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.171357 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.171267    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92387b00-938f-451a-8aac-776a13df9d61-config-out\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.171828 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.171804    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92387b00-938f-451a-8aac-776a13df9d61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.171992 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.171973    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/92387b00-938f-451a-8aac-776a13df9d61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.174864 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.174845    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crfjk\" (UniqueName: \"kubernetes.io/projected/92387b00-938f-451a-8aac-776a13df9d61-kube-api-access-crfjk\") pod \"prometheus-k8s-0\" (UID: \"92387b00-938f-451a-8aac-776a13df9d61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.320590 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.320514    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:49:29.471046 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.471018    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 15:49:29.472663 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:49:29.472637    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92387b00_938f_451a_8aac_776a13df9d61.slice/crio-a3967400fa3df2e77ebeb3b7689d916dcde0f6f8efa101bebe73286773b0c7bd WatchSource:0}: Error finding container a3967400fa3df2e77ebeb3b7689d916dcde0f6f8efa101bebe73286773b0c7bd: Status 404 returned error can't find the container with id a3967400fa3df2e77ebeb3b7689d916dcde0f6f8efa101bebe73286773b0c7bd
Jul 14 15:49:29.944073 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.944035    2560 generic.go:356] "Generic (PLEG): container finished" podID="92387b00-938f-451a-8aac-776a13df9d61" containerID="8f08ee18753a1b91e6af4c7b11a2db41ffe6cb5ebeaff042ace41da19bd6a4b4" exitCode=0
Jul 14 15:49:29.944576 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.944115    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92387b00-938f-451a-8aac-776a13df9d61","Type":"ContainerDied","Data":"8f08ee18753a1b91e6af4c7b11a2db41ffe6cb5ebeaff042ace41da19bd6a4b4"}
Jul 14 15:49:29.944576 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:29.944212    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92387b00-938f-451a-8aac-776a13df9d61","Type":"ContainerStarted","Data":"a3967400fa3df2e77ebeb3b7689d916dcde0f6f8efa101bebe73286773b0c7bd"}
Jul 14 15:49:30.952723 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:30.952634    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m" event={"ID":"1cc47fc7-d572-4281-9e26-71c1ef812bea","Type":"ContainerStarted","Data":"4ad549380216943def8d4772d1e88f26377951b5b64ce790f620eb270c987d22"}
Jul 14 15:49:30.953129 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:30.952737    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m" event={"ID":"1cc47fc7-d572-4281-9e26-71c1ef812bea","Type":"ContainerStarted","Data":"08031ce0679649a106db4147b5aee73aafdff495fb432bf82f38c0e335dc52f3"}
Jul 14 15:49:30.953129 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:30.952764    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m" event={"ID":"1cc47fc7-d572-4281-9e26-71c1ef812bea","Type":"ContainerStarted","Data":"5faea0246e42a3ca3a1e3697c71edb2613afa3080a27b5f073743cec390be5ee"}
Jul 14 15:49:30.955853 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:30.955823    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92387b00-938f-451a-8aac-776a13df9d61","Type":"ContainerStarted","Data":"1e922a368d2f6e4eb1bc1e58c74ef278fd03ac59c31eb67a954eb7b3838bf12e"}
Jul 14 15:49:30.955954 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:30.955894    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92387b00-938f-451a-8aac-776a13df9d61","Type":"ContainerStarted","Data":"0dda10170340b7d8e9e23d72e96d5618f13ef63e226d0157be8c6fbd7d05f356"}
Jul 14 15:49:30.955954 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:30.955934    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92387b00-938f-451a-8aac-776a13df9d61","Type":"ContainerStarted","Data":"bb4861d4b8fcb8ca04c08d6c4c2a7e077ee5e9265cd7b46cbc867e0bde70f964"}
Jul 14 15:49:30.956050 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:30.955978    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92387b00-938f-451a-8aac-776a13df9d61","Type":"ContainerStarted","Data":"b6f661f2c9708784a59a94b18f18c513a3b48e3a7cf06bf4ed4e43bbaa0db298"}
Jul 14 15:49:30.956050 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:30.956009    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92387b00-938f-451a-8aac-776a13df9d61","Type":"ContainerStarted","Data":"9e435e88ac979000d22431d04e3d3cbd799c908fc39f4319340c1efb8b992c3e"}
Jul 14 15:49:30.956050 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:30.956033    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"92387b00-938f-451a-8aac-776a13df9d61","Type":"ContainerStarted","Data":"0961ad1d848e4a72cfcb1dd1e66d10eca36ea4a59d2243eb78c89739d666d5b1"}
Jul 14 15:49:30.975203 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:30.974997    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6b54b766fc-c5k2m" podStartSLOduration=1.809283596 podStartE2EDuration="3.974977418s" totalImagesPullingTime="2.165693822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:49:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:49:30.973780108 +0000 UTC m=+154.493147530" watchObservedRunningTime="2026-07-14 15:49:30.974977418 +0000 UTC m=+154.494344840"
Jul 14 15:49:31.004333 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:31.004276    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.985286892 podStartE2EDuration="3.004259506s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.972614ms" isStatefulPod=false podCreationTimestamp="2026-07-14 15:49:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:49:31.002234046 +0000 UTC m=+154.521601466" watchObservedRunningTime="2026-07-14 15:49:31.004259506 +0000 UTC m=+154.523626927"
Jul 14 15:49:34.320627 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:49:34.320578    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:50:29.321038 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:50:29.321003    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:50:29.337627 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:50:29.337601    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:50:30.212433 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:50:30.212408    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 15:51:56.875345 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:51:56.875320    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 15:51:56.882426 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:51:56.882385    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 15:51:56.883332 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:51:56.883300    2560 kubelet.go:1648] "Image garbage collection succeeded"
Jul 14 15:52:37.565411 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.565367    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7m8hv"]
Jul 14 15:52:37.568871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.568851    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv"
Jul 14 15:52:37.571500 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.571478    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-r2hlf\""
Jul 14 15:52:37.571699 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.571686    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 14 15:52:37.572683 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.572668    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 14 15:52:37.581192 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.581175    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7m8hv"]
Jul 14 15:52:37.648797 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.648769    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5939a333-1966-49a1-b89e-5eeeb5e607d8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7m8hv\" (UID: \"5939a333-1966-49a1-b89e-5eeeb5e607d8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv"
Jul 14 15:52:37.648955 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.648831    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqksw\" (UniqueName: \"kubernetes.io/projected/5939a333-1966-49a1-b89e-5eeeb5e607d8-kube-api-access-rqksw\") pod \"cert-manager-webhook-6486ffd997-7m8hv\" (UID: \"5939a333-1966-49a1-b89e-5eeeb5e607d8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv"
Jul 14 15:52:37.749345 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.749315    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5939a333-1966-49a1-b89e-5eeeb5e607d8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7m8hv\" (UID: \"5939a333-1966-49a1-b89e-5eeeb5e607d8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv"
Jul 14 15:52:37.749535 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.749371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqksw\" (UniqueName: \"kubernetes.io/projected/5939a333-1966-49a1-b89e-5eeeb5e607d8-kube-api-access-rqksw\") pod \"cert-manager-webhook-6486ffd997-7m8hv\" (UID: \"5939a333-1966-49a1-b89e-5eeeb5e607d8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv"
Jul 14 15:52:37.762145 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.762106    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5939a333-1966-49a1-b89e-5eeeb5e607d8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7m8hv\" (UID: \"5939a333-1966-49a1-b89e-5eeeb5e607d8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv"
Jul 14 15:52:37.762243 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.762219    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqksw\" (UniqueName: \"kubernetes.io/projected/5939a333-1966-49a1-b89e-5eeeb5e607d8-kube-api-access-rqksw\") pod \"cert-manager-webhook-6486ffd997-7m8hv\" (UID: \"5939a333-1966-49a1-b89e-5eeeb5e607d8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv"
Jul 14 15:52:37.889551 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:37.889520    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv"
Jul 14 15:52:38.040566 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:38.040545    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7m8hv"]
Jul 14 15:52:38.042465 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:52:38.042427    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5939a333_1966_49a1_b89e_5eeeb5e607d8.slice/crio-6aa78addb57ad86057536a6af47c10792554fb6e4681054d3534615574097692 WatchSource:0}: Error finding container 6aa78addb57ad86057536a6af47c10792554fb6e4681054d3534615574097692: Status 404 returned error can't find the container with id 6aa78addb57ad86057536a6af47c10792554fb6e4681054d3534615574097692
Jul 14 15:52:38.044765 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:38.044742    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 15:52:38.705505 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:38.705472    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv" event={"ID":"5939a333-1966-49a1-b89e-5eeeb5e607d8","Type":"ContainerStarted","Data":"6aa78addb57ad86057536a6af47c10792554fb6e4681054d3534615574097692"}
Jul 14 15:52:41.721900 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:41.721815    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv" event={"ID":"5939a333-1966-49a1-b89e-5eeeb5e607d8","Type":"ContainerStarted","Data":"3eb874801144d64be8359ab901e73b0ec2b20a332c5e338efede5bd1f9a3725f"}
Jul 14 15:52:41.722247 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:41.721945    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv"
Jul 14 15:52:41.740169 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:41.740121    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv" podStartSLOduration=1.419702804 podStartE2EDuration="4.740105891s" totalImagesPullingTime="3.320403087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:52:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:52:41.739359112 +0000 UTC m=+345.258726535" watchObservedRunningTime="2026-07-14 15:52:41.740105891 +0000 UTC m=+345.259473312"
Jul 14 15:52:47.727871 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:47.727792    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7m8hv"
Jul 14 15:52:51.197045 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.197009    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6"]
Jul 14 15:52:51.199997 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.199976    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6"
Jul 14 15:52:51.202802 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.202782    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 14 15:52:51.203625 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.203600    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 14 15:52:51.203730 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.203624    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-sffqp\""
Jul 14 15:52:51.209021 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.209001    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6"]
Jul 14 15:52:51.273584 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.273561    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghtmx\" (UniqueName: \"kubernetes.io/projected/917b405c-b5f6-49ff-be48-41b30189b2c6-kube-api-access-ghtmx\") pod \"openshift-lws-operator-fd8ccff4c-rtfw6\" (UID: \"917b405c-b5f6-49ff-be48-41b30189b2c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6"
Jul 14 15:52:51.273728 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.273592    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/917b405c-b5f6-49ff-be48-41b30189b2c6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rtfw6\" (UID: \"917b405c-b5f6-49ff-be48-41b30189b2c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6"
Jul 14 15:52:51.374604 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.374569    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghtmx\" (UniqueName: \"kubernetes.io/projected/917b405c-b5f6-49ff-be48-41b30189b2c6-kube-api-access-ghtmx\") pod \"openshift-lws-operator-fd8ccff4c-rtfw6\" (UID: \"917b405c-b5f6-49ff-be48-41b30189b2c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6"
Jul 14 15:52:51.374766 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.374613    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/917b405c-b5f6-49ff-be48-41b30189b2c6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rtfw6\" (UID: \"917b405c-b5f6-49ff-be48-41b30189b2c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6"
Jul 14 15:52:51.374971 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.374955    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/917b405c-b5f6-49ff-be48-41b30189b2c6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rtfw6\" (UID: \"917b405c-b5f6-49ff-be48-41b30189b2c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6"
Jul 14 15:52:51.383199 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.383170    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghtmx\" (UniqueName: \"kubernetes.io/projected/917b405c-b5f6-49ff-be48-41b30189b2c6-kube-api-access-ghtmx\") pod \"openshift-lws-operator-fd8ccff4c-rtfw6\" (UID: \"917b405c-b5f6-49ff-be48-41b30189b2c6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6"
Jul 14 15:52:51.510157 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.510062    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6"
Jul 14 15:52:51.639742 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.639719    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6"]
Jul 14 15:52:51.642098 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:52:51.642070    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod917b405c_b5f6_49ff_be48_41b30189b2c6.slice/crio-40a40c1295067c58db0bb4adeea0bb63cabb6ccc3aff70946adbe2ae28bfdeca WatchSource:0}: Error finding container 40a40c1295067c58db0bb4adeea0bb63cabb6ccc3aff70946adbe2ae28bfdeca: Status 404 returned error can't find the container with id 40a40c1295067c58db0bb4adeea0bb63cabb6ccc3aff70946adbe2ae28bfdeca
Jul 14 15:52:51.762666 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:51.762585    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6" event={"ID":"917b405c-b5f6-49ff-be48-41b30189b2c6","Type":"ContainerStarted","Data":"40a40c1295067c58db0bb4adeea0bb63cabb6ccc3aff70946adbe2ae28bfdeca"}
Jul 14 15:52:54.778976 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:54.778893    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6" event={"ID":"917b405c-b5f6-49ff-be48-41b30189b2c6","Type":"ContainerStarted","Data":"1f6afe97de6bf927f2f28cd1d47b4679edbc23ddb6743611eb52826c71cbc7e5"}
Jul 14 15:52:54.802093 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:52:54.802046    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rtfw6" podStartSLOduration=1.074808466 podStartE2EDuration="3.802028541s" totalImagesPullingTime="2.727220075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:52:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:52:54.800344704 +0000 UTC m=+358.319712127" watchObservedRunningTime="2026-07-14 15:52:54.802028541 +0000 UTC m=+358.321395962"
Jul 14 15:53:03.793099 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.793065    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"]
Jul 14 15:53:03.796533 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.796511    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.800956 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.800935    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4cwj7\""
Jul 14 15:53:03.801077 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.800953    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 14 15:53:03.801077 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.800963    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 14 15:53:03.801174 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.801123    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 14 15:53:03.806699 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.806682    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"]
Jul 14 15:53:03.889200 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.889161    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wqfm\" (UniqueName: \"kubernetes.io/projected/8a453c4c-a2ac-49e9-b663-59217c2fe45c-kube-api-access-4wqfm\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.889200 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.889215    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a453c4c-a2ac-49e9-b663-59217c2fe45c-metrics-cert\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.889546 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.889252    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a453c4c-a2ac-49e9-b663-59217c2fe45c-manager-config\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.889546 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.889299    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a453c4c-a2ac-49e9-b663-59217c2fe45c-cert\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.990575 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.990536    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a453c4c-a2ac-49e9-b663-59217c2fe45c-cert\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.990751 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.990612    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wqfm\" (UniqueName: \"kubernetes.io/projected/8a453c4c-a2ac-49e9-b663-59217c2fe45c-kube-api-access-4wqfm\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.990751 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.990652    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a453c4c-a2ac-49e9-b663-59217c2fe45c-metrics-cert\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.990751 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.990682    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a453c4c-a2ac-49e9-b663-59217c2fe45c-manager-config\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.991392 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.991318    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a453c4c-a2ac-49e9-b663-59217c2fe45c-manager-config\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.993137 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.993114    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a453c4c-a2ac-49e9-b663-59217c2fe45c-cert\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.993228 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.993152    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a453c4c-a2ac-49e9-b663-59217c2fe45c-metrics-cert\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:03.998764 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:03.998739    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wqfm\" (UniqueName: \"kubernetes.io/projected/8a453c4c-a2ac-49e9-b663-59217c2fe45c-kube-api-access-4wqfm\") pod \"lws-controller-manager-658b99d874-hlsdr\" (UID: \"8a453c4c-a2ac-49e9-b663-59217c2fe45c\") " pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:04.106050 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:04.106020    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:04.239036 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:04.239006    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"]
Jul 14 15:53:04.242810 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:53:04.242782    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a453c4c_a2ac_49e9_b663_59217c2fe45c.slice/crio-739e862f9c8eeee50abb7b25a4143db28ebfb34f3df0791dd17235e478d0b540 WatchSource:0}: Error finding container 739e862f9c8eeee50abb7b25a4143db28ebfb34f3df0791dd17235e478d0b540: Status 404 returned error can't find the container with id 739e862f9c8eeee50abb7b25a4143db28ebfb34f3df0791dd17235e478d0b540
Jul 14 15:53:04.822431 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:04.822391    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr" event={"ID":"8a453c4c-a2ac-49e9-b663-59217c2fe45c","Type":"ContainerStarted","Data":"739e862f9c8eeee50abb7b25a4143db28ebfb34f3df0791dd17235e478d0b540"}
Jul 14 15:53:05.828340 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:05.828256    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr" event={"ID":"8a453c4c-a2ac-49e9-b663-59217c2fe45c","Type":"ContainerStarted","Data":"94bc414c5a5079934a0a54e383e37c7c0114d88b8b823806dcd6c295e5e6952a"}
Jul 14 15:53:05.828340 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:05.828319    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:05.847865 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:05.847816    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr" podStartSLOduration=1.639238341 podStartE2EDuration="2.847799528s" totalImagesPullingTime="1.208561187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:53:05.845766854 +0000 UTC m=+369.365134276" watchObservedRunningTime="2026-07-14 15:53:05.847799528 +0000 UTC m=+369.367166949"
Jul 14 15:53:14.378325 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.378293    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"]
Jul 14 15:53:14.421924 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.421872    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"]
Jul 14 15:53:14.422106 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.421995    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.426463 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.426421    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 14 15:53:14.429729 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.429706    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 14 15:53:14.429853 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.429816    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-klszc\""
Jul 14 15:53:14.430637 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.430611    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 14 15:53:14.430750 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.430716    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 14 15:53:14.488229 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.488198    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c5c6b30-088a-42a3-bedd-a386ae2657a7-tmp\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.488414 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.488244    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c5c6b30-088a-42a3-bedd-a386ae2657a7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.488414 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.488273    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmpzz\" (UniqueName: \"kubernetes.io/projected/8c5c6b30-088a-42a3-bedd-a386ae2657a7-kube-api-access-vmpzz\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.488414 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.488342    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c5c6b30-088a-42a3-bedd-a386ae2657a7-manifests\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.488597 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.488464    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c5c6b30-088a-42a3-bedd-a386ae2657a7-webhook-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.589534 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.589496    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c5c6b30-088a-42a3-bedd-a386ae2657a7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.589534 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.589539    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmpzz\" (UniqueName: \"kubernetes.io/projected/8c5c6b30-088a-42a3-bedd-a386ae2657a7-kube-api-access-vmpzz\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.589797 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.589564    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c5c6b30-088a-42a3-bedd-a386ae2657a7-manifests\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.589797 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.589616    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c5c6b30-088a-42a3-bedd-a386ae2657a7-webhook-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.589797 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.589714    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c5c6b30-088a-42a3-bedd-a386ae2657a7-tmp\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.590172 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.590144    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c5c6b30-088a-42a3-bedd-a386ae2657a7-manifests\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.592186 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.592163    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c5c6b30-088a-42a3-bedd-a386ae2657a7-webhook-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.592461 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.592425    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c5c6b30-088a-42a3-bedd-a386ae2657a7-tmp\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.592571 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.592555    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c5c6b30-088a-42a3-bedd-a386ae2657a7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.603301 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.603274    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmpzz\" (UniqueName: \"kubernetes.io/projected/8c5c6b30-088a-42a3-bedd-a386ae2657a7-kube-api-access-vmpzz\") pod \"opendatahub-operator-controller-manager-7486689d9d-h8khl\" (UID: \"8c5c6b30-088a-42a3-bedd-a386ae2657a7\") " pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.734568 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.734486    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:14.900392 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:14.900368    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"]
Jul 14 15:53:14.902144 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:53:14.902113    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5c6b30_088a_42a3_bedd_a386ae2657a7.slice/crio-eb68c520482e33b027fb6bdcc1407b024f2730d59775eb2c6d72316ab1c4c1dd WatchSource:0}: Error finding container eb68c520482e33b027fb6bdcc1407b024f2730d59775eb2c6d72316ab1c4c1dd: Status 404 returned error can't find the container with id eb68c520482e33b027fb6bdcc1407b024f2730d59775eb2c6d72316ab1c4c1dd
Jul 14 15:53:15.871201 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:15.871169    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl" event={"ID":"8c5c6b30-088a-42a3-bedd-a386ae2657a7","Type":"ContainerStarted","Data":"eb68c520482e33b027fb6bdcc1407b024f2730d59775eb2c6d72316ab1c4c1dd"}
Jul 14 15:53:16.835546 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:16.835513    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-658b99d874-hlsdr"
Jul 14 15:53:18.889842 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:18.889805    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl" event={"ID":"8c5c6b30-088a-42a3-bedd-a386ae2657a7","Type":"ContainerStarted","Data":"8cfac28c93f8d2170c94fed5fe8e08b38085a2fb161fe7f3681f267044c40482"}
Jul 14 15:53:19.895855 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:19.895817    2560 generic.go:356] "Generic (PLEG): container finished" podID="8c5c6b30-088a-42a3-bedd-a386ae2657a7" containerID="8cfac28c93f8d2170c94fed5fe8e08b38085a2fb161fe7f3681f267044c40482" exitCode=0
Jul 14 15:53:19.896215 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:19.895862    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl" event={"ID":"8c5c6b30-088a-42a3-bedd-a386ae2657a7","Type":"ContainerDied","Data":"8cfac28c93f8d2170c94fed5fe8e08b38085a2fb161fe7f3681f267044c40482"}
Jul 14 15:53:20.902500 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:20.902436    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl" event={"ID":"8c5c6b30-088a-42a3-bedd-a386ae2657a7","Type":"ContainerStarted","Data":"89fcb27eda9ba07f15d1400db76441eb9121b062537a5d3d88e28df9f14be5f5"}
Jul 14 15:53:20.902840 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:20.902613    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:20.924439 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:20.924396    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl" podStartSLOduration=1.981162636 podStartE2EDuration="6.924368545s" totalImagesPullingTime="3.887638391s" totalInitContainerRuntime="1.055567518s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:53:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:53:20.923232102 +0000 UTC m=+384.442599536" watchObservedRunningTime="2026-07-14 15:53:20.924368545 +0000 UTC m=+384.443735966"
Jul 14 15:53:31.909583 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:31.909555    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7486689d9d-h8khl"
Jul 14 15:53:34.816668 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:34.816634    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"]
Jul 14 15:53:34.824308 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:34.824284    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"
Jul 14 15:53:34.827168 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:34.827150    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 14 15:53:34.827343 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:34.827326    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 14 15:53:34.828165 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:34.828146    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 14 15:53:34.828312 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:34.828213    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8mx9f\""
Jul 14 15:53:34.828403 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:34.828363    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 14 15:53:34.838325 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:34.838303    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"]
Jul 14 15:53:34.969880 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:34.969844    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9230b14d-42a8-4647-aba8-10bd740658c0-tls-certs\") pod \"kube-auth-proxy-85568bb86c-99mrc\" (UID: \"9230b14d-42a8-4647-aba8-10bd740658c0\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"
Jul 14 15:53:34.970104 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:34.969898    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2xd7\" (UniqueName: \"kubernetes.io/projected/9230b14d-42a8-4647-aba8-10bd740658c0-kube-api-access-n2xd7\") pod \"kube-auth-proxy-85568bb86c-99mrc\" (UID: \"9230b14d-42a8-4647-aba8-10bd740658c0\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"
Jul 14 15:53:34.970104 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:34.970041    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9230b14d-42a8-4647-aba8-10bd740658c0-tmp\") pod \"kube-auth-proxy-85568bb86c-99mrc\" (UID: \"9230b14d-42a8-4647-aba8-10bd740658c0\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"
Jul 14 15:53:35.071332 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:35.071252    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9230b14d-42a8-4647-aba8-10bd740658c0-tmp\") pod \"kube-auth-proxy-85568bb86c-99mrc\" (UID: \"9230b14d-42a8-4647-aba8-10bd740658c0\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"
Jul 14 15:53:35.071499 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:35.071353    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9230b14d-42a8-4647-aba8-10bd740658c0-tls-certs\") pod \"kube-auth-proxy-85568bb86c-99mrc\" (UID: \"9230b14d-42a8-4647-aba8-10bd740658c0\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"
Jul 14 15:53:35.071499 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:35.071383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2xd7\" (UniqueName: \"kubernetes.io/projected/9230b14d-42a8-4647-aba8-10bd740658c0-kube-api-access-n2xd7\") pod \"kube-auth-proxy-85568bb86c-99mrc\" (UID: \"9230b14d-42a8-4647-aba8-10bd740658c0\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"
Jul 14 15:53:35.073672 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:35.073639    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9230b14d-42a8-4647-aba8-10bd740658c0-tmp\") pod \"kube-auth-proxy-85568bb86c-99mrc\" (UID: \"9230b14d-42a8-4647-aba8-10bd740658c0\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"
Jul 14 15:53:35.073897 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:35.073881    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9230b14d-42a8-4647-aba8-10bd740658c0-tls-certs\") pod \"kube-auth-proxy-85568bb86c-99mrc\" (UID: \"9230b14d-42a8-4647-aba8-10bd740658c0\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"
Jul 14 15:53:35.078899 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:35.078876    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2xd7\" (UniqueName: \"kubernetes.io/projected/9230b14d-42a8-4647-aba8-10bd740658c0-kube-api-access-n2xd7\") pod \"kube-auth-proxy-85568bb86c-99mrc\" (UID: \"9230b14d-42a8-4647-aba8-10bd740658c0\") " pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"
Jul 14 15:53:35.134319 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:35.134294    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"
Jul 14 15:53:35.286188 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:35.286156    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85568bb86c-99mrc"]
Jul 14 15:53:35.289849 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:53:35.289816    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9230b14d_42a8_4647_aba8_10bd740658c0.slice/crio-be6393b777993bced0e8cb6cece4a3fbb4534f50a6339d432cd078fefb12fc11 WatchSource:0}: Error finding container be6393b777993bced0e8cb6cece4a3fbb4534f50a6339d432cd078fefb12fc11: Status 404 returned error can't find the container with id be6393b777993bced0e8cb6cece4a3fbb4534f50a6339d432cd078fefb12fc11
Jul 14 15:53:35.970483 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:35.970435    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc" event={"ID":"9230b14d-42a8-4647-aba8-10bd740658c0","Type":"ContainerStarted","Data":"be6393b777993bced0e8cb6cece4a3fbb4534f50a6339d432cd078fefb12fc11"}
Jul 14 15:53:38.986233 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:38.986200    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc" event={"ID":"9230b14d-42a8-4647-aba8-10bd740658c0","Type":"ContainerStarted","Data":"4e5bfd64c7fb837cea161a66eb4ce10cbbad0907c8e482b829eb74b985d8b41d"}
Jul 14 15:53:39.005517 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:53:39.005475    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85568bb86c-99mrc" podStartSLOduration=1.7398207380000001 podStartE2EDuration="5.005438522s" totalImagesPullingTime="3.265617784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:53:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:53:39.004113921 +0000 UTC m=+402.523481343" watchObservedRunningTime="2026-07-14 15:53:39.005438522 +0000 UTC m=+402.524805942"
Jul 14 15:54:43.923476 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.923387    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fbb5fd7-x9mp7"]
Jul 14 15:54:43.925911 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.925891    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:43.928494 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.928464    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 14 15:54:43.928616 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.928558    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 14 15:54:43.928686 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.928474    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 14 15:54:43.928686 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.928608    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 14 15:54:43.929705 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.929686    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 14 15:54:43.929812 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.929706    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 14 15:54:43.929812 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.929798    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 14 15:54:43.930121 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.930106    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qvsqv\""
Jul 14 15:54:43.934530 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.934513    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 14 15:54:43.943418 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.943400    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fbb5fd7-x9mp7"]
Jul 14 15:54:43.972796 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.972772    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-service-ca\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:43.972897 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.972817    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-trusted-ca-bundle\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:43.972897 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.972854    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-console-config\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:43.972970 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.972904    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-oauth-serving-cert\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:43.972970 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.972945    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jpkq\" (UniqueName: \"kubernetes.io/projected/8058c125-7ec1-4eaf-b480-d979fd3c394a-kube-api-access-7jpkq\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:43.973071 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.972992    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8058c125-7ec1-4eaf-b480-d979fd3c394a-console-serving-cert\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:43.973071 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:43.973014    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8058c125-7ec1-4eaf-b480-d979fd3c394a-console-oauth-config\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.073676 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.073648    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8058c125-7ec1-4eaf-b480-d979fd3c394a-console-oauth-config\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.073796 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.073693    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-service-ca\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.073796 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.073759    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-trusted-ca-bundle\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.073885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.073807    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-console-config\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.073885 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.073834    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-oauth-serving-cert\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.073980 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.073953    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jpkq\" (UniqueName: \"kubernetes.io/projected/8058c125-7ec1-4eaf-b480-d979fd3c394a-kube-api-access-7jpkq\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.074126 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.074035    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8058c125-7ec1-4eaf-b480-d979fd3c394a-console-serving-cert\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.074619 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.074595    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-service-ca\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.074712 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.074615    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-trusted-ca-bundle\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.074712 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.074668    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-oauth-serving-cert\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.074877 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.074854    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8058c125-7ec1-4eaf-b480-d979fd3c394a-console-config\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.076191 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.076169    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8058c125-7ec1-4eaf-b480-d979fd3c394a-console-oauth-config\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.076582 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.076566    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8058c125-7ec1-4eaf-b480-d979fd3c394a-console-serving-cert\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.086308 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.086278    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jpkq\" (UniqueName: \"kubernetes.io/projected/8058c125-7ec1-4eaf-b480-d979fd3c394a-kube-api-access-7jpkq\") pod \"console-7fbb5fd7-x9mp7\" (UID: \"8058c125-7ec1-4eaf-b480-d979fd3c394a\") " pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.236037 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.235960    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:44.372675 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:44.372652    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fbb5fd7-x9mp7"]
Jul 14 15:54:44.374922 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:54:44.374896    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8058c125_7ec1_4eaf_b480_d979fd3c394a.slice/crio-c3e1afdbdfa58fd280c78290e7894553684339729053ce2c6794c9e8842a7423 WatchSource:0}: Error finding container c3e1afdbdfa58fd280c78290e7894553684339729053ce2c6794c9e8842a7423: Status 404 returned error can't find the container with id c3e1afdbdfa58fd280c78290e7894553684339729053ce2c6794c9e8842a7423
Jul 14 15:54:45.285619 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:45.285586    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fbb5fd7-x9mp7" event={"ID":"8058c125-7ec1-4eaf-b480-d979fd3c394a","Type":"ContainerStarted","Data":"c433d7dff2b5ff9f28db527dfa7d4ed793ba14f9c1a04ae3158ee391f493a583"}
Jul 14 15:54:45.286000 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:45.285637    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fbb5fd7-x9mp7" event={"ID":"8058c125-7ec1-4eaf-b480-d979fd3c394a","Type":"ContainerStarted","Data":"c3e1afdbdfa58fd280c78290e7894553684339729053ce2c6794c9e8842a7423"}
Jul 14 15:54:45.306819 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:45.306771    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fbb5fd7-x9mp7" podStartSLOduration=2.306756039 podStartE2EDuration="2.306756039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:54:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:54:45.30549464 +0000 UTC m=+468.824862064" watchObservedRunningTime="2026-07-14 15:54:45.306756039 +0000 UTC m=+468.826123528"
Jul 14 15:54:54.236122 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:54.236083    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:54.236614 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:54.236160    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:54.240955 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:54.240933    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:54.330088 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:54.330064    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fbb5fd7-x9mp7"
Jul 14 15:54:56.910709 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:54:56.910677    2560 scope.go:117] "RemoveContainer" containerID="02e7944033f313906c3260efb25cc0ecae9dc333422adfba0c3a87292751b822"
Jul 14 15:55:11.731376 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:11.731346    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj"]
Jul 14 15:55:11.734545 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:11.734530    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj"
Jul 14 15:55:11.737058 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:11.737035    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 14 15:55:11.737200 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:11.737036    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 14 15:55:11.737792 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:11.737772    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 14 15:55:11.737880 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:11.737772    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7kqtp\""
Jul 14 15:55:11.746925 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:11.746905    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj"]
Jul 14 15:55:11.828879 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:11.828842    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwh6h\" (UniqueName: \"kubernetes.io/projected/e4ee7d3a-ab03-47c7-851f-93600b3ff276-kube-api-access-cwh6h\") pod \"dns-operator-controller-manager-864dd78c4c-8jwwj\" (UID: \"e4ee7d3a-ab03-47c7-851f-93600b3ff276\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj"
Jul 14 15:55:11.929433 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:11.929401    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwh6h\" (UniqueName: \"kubernetes.io/projected/e4ee7d3a-ab03-47c7-851f-93600b3ff276-kube-api-access-cwh6h\") pod \"dns-operator-controller-manager-864dd78c4c-8jwwj\" (UID: \"e4ee7d3a-ab03-47c7-851f-93600b3ff276\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj"
Jul 14 15:55:11.939320 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:11.939289    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwh6h\" (UniqueName: \"kubernetes.io/projected/e4ee7d3a-ab03-47c7-851f-93600b3ff276-kube-api-access-cwh6h\") pod \"dns-operator-controller-manager-864dd78c4c-8jwwj\" (UID: \"e4ee7d3a-ab03-47c7-851f-93600b3ff276\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj"
Jul 14 15:55:12.044646 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:12.044576    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj"
Jul 14 15:55:12.177698 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:12.177674    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj"]
Jul 14 15:55:12.179866 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:55:12.179836    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ee7d3a_ab03_47c7_851f_93600b3ff276.slice/crio-c64fb39ce1ff079b94765098cb20ba1908d75951079791407e9b9c60cbcb52f3 WatchSource:0}: Error finding container c64fb39ce1ff079b94765098cb20ba1908d75951079791407e9b9c60cbcb52f3: Status 404 returned error can't find the container with id c64fb39ce1ff079b94765098cb20ba1908d75951079791407e9b9c60cbcb52f3
Jul 14 15:55:12.407062 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:12.407028    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj" event={"ID":"e4ee7d3a-ab03-47c7-851f-93600b3ff276","Type":"ContainerStarted","Data":"c64fb39ce1ff079b94765098cb20ba1908d75951079791407e9b9c60cbcb52f3"}
Jul 14 15:55:14.185771 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.185743    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"]
Jul 14 15:55:14.188602 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.188580    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:14.191128 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.191101    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 14 15:55:14.191241 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.191155    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 14 15:55:14.191241 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.191176    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wt668\""
Jul 14 15:55:14.197677 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.197655    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"]
Jul 14 15:55:14.251829 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.251797    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9bc7\" (UniqueName: \"kubernetes.io/projected/bfaff471-4b4c-4986-a88d-7b94d15f86da-kube-api-access-r9bc7\") pod \"kuadrant-console-plugin-66b5774985-z72nv\" (UID: \"bfaff471-4b4c-4986-a88d-7b94d15f86da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:14.252012 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.251867    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfaff471-4b4c-4986-a88d-7b94d15f86da-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z72nv\" (UID: \"bfaff471-4b4c-4986-a88d-7b94d15f86da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:14.252012 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.251988    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bfaff471-4b4c-4986-a88d-7b94d15f86da-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z72nv\" (UID: \"bfaff471-4b4c-4986-a88d-7b94d15f86da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:14.352562 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.352522    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfaff471-4b4c-4986-a88d-7b94d15f86da-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z72nv\" (UID: \"bfaff471-4b4c-4986-a88d-7b94d15f86da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:14.352737 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.352648    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bfaff471-4b4c-4986-a88d-7b94d15f86da-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z72nv\" (UID: \"bfaff471-4b4c-4986-a88d-7b94d15f86da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:14.352737 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:55:14.352659    2560 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 14 15:55:14.352737 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.352700    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9bc7\" (UniqueName: \"kubernetes.io/projected/bfaff471-4b4c-4986-a88d-7b94d15f86da-kube-api-access-r9bc7\") pod \"kuadrant-console-plugin-66b5774985-z72nv\" (UID: \"bfaff471-4b4c-4986-a88d-7b94d15f86da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:14.352737 ip-10-0-135-20 kubenswrapper[2560]: E0714 15:55:14.352734    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfaff471-4b4c-4986-a88d-7b94d15f86da-plugin-serving-cert podName:bfaff471-4b4c-4986-a88d-7b94d15f86da nodeName:}" failed. No retries permitted until 2026-07-14 15:55:14.852711934 +0000 UTC m=+498.372079382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/bfaff471-4b4c-4986-a88d-7b94d15f86da-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-z72nv" (UID: "bfaff471-4b4c-4986-a88d-7b94d15f86da") : secret "plugin-serving-cert" not found
Jul 14 15:55:14.353394 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.353372    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bfaff471-4b4c-4986-a88d-7b94d15f86da-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z72nv\" (UID: \"bfaff471-4b4c-4986-a88d-7b94d15f86da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:14.387168 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.387120    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9bc7\" (UniqueName: \"kubernetes.io/projected/bfaff471-4b4c-4986-a88d-7b94d15f86da-kube-api-access-r9bc7\") pod \"kuadrant-console-plugin-66b5774985-z72nv\" (UID: \"bfaff471-4b4c-4986-a88d-7b94d15f86da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:14.856696 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.856660    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfaff471-4b4c-4986-a88d-7b94d15f86da-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z72nv\" (UID: \"bfaff471-4b4c-4986-a88d-7b94d15f86da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:14.859200 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:14.859177    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfaff471-4b4c-4986-a88d-7b94d15f86da-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z72nv\" (UID: \"bfaff471-4b4c-4986-a88d-7b94d15f86da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:15.100646 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:15.100614    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"
Jul 14 15:55:15.239865 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:15.239841    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv"]
Jul 14 15:55:15.241586 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:55:15.241562    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfaff471_4b4c_4986_a88d_7b94d15f86da.slice/crio-99a14a6d5a21883fea0fc6a3848de7b9fdd994a7c7bb37ca0fd8f1fe36b90716 WatchSource:0}: Error finding container 99a14a6d5a21883fea0fc6a3848de7b9fdd994a7c7bb37ca0fd8f1fe36b90716: Status 404 returned error can't find the container with id 99a14a6d5a21883fea0fc6a3848de7b9fdd994a7c7bb37ca0fd8f1fe36b90716
Jul 14 15:55:15.424401 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:15.424316    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv" event={"ID":"bfaff471-4b4c-4986-a88d-7b94d15f86da","Type":"ContainerStarted","Data":"99a14a6d5a21883fea0fc6a3848de7b9fdd994a7c7bb37ca0fd8f1fe36b90716"}
Jul 14 15:55:15.425746 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:15.425723    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj" event={"ID":"e4ee7d3a-ab03-47c7-851f-93600b3ff276","Type":"ContainerStarted","Data":"cbe521b782ffc1c0a31d317d209487eb9bccad49a07567b6791439bd2b44a5aa"}
Jul 14 15:55:15.425857 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:15.425839    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj"
Jul 14 15:55:15.447203 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:15.447155    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj" podStartSLOduration=2.151952467 podStartE2EDuration="4.447138108s" totalImagesPullingTime="2.295185641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:55:15.445594209 +0000 UTC m=+498.964961697" watchObservedRunningTime="2026-07-14 15:55:15.447138108 +0000 UTC m=+498.966505531"
Jul 14 15:55:25.090747 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:25.090717    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb"]
Jul 14 15:55:25.096629 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:25.096601    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb"
Jul 14 15:55:25.099347 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:25.099328    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hh6v7\""
Jul 14 15:55:25.104887 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:25.104867    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb"]
Jul 14 15:55:25.258701 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:25.258660    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-797nq\" (UniqueName: \"kubernetes.io/projected/f93a3446-61bb-4619-9c7e-96b51398ff73-kube-api-access-797nq\") pod \"limitador-operator-controller-manager-859d7b78bd-nv6pb\" (UID: \"f93a3446-61bb-4619-9c7e-96b51398ff73\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb"
Jul 14 15:55:25.360354 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:25.360283    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-797nq\" (UniqueName: \"kubernetes.io/projected/f93a3446-61bb-4619-9c7e-96b51398ff73-kube-api-access-797nq\") pod \"limitador-operator-controller-manager-859d7b78bd-nv6pb\" (UID: \"f93a3446-61bb-4619-9c7e-96b51398ff73\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb"
Jul 14 15:55:25.374424 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:25.374396    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-797nq\" (UniqueName: \"kubernetes.io/projected/f93a3446-61bb-4619-9c7e-96b51398ff73-kube-api-access-797nq\") pod \"limitador-operator-controller-manager-859d7b78bd-nv6pb\" (UID: \"f93a3446-61bb-4619-9c7e-96b51398ff73\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb"
Jul 14 15:55:25.410750 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:25.410721    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb"
Jul 14 15:55:26.434735 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:26.434702    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8jwwj"
Jul 14 15:55:38.696319 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:38.696294    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb"]
Jul 14 15:55:38.698501 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:55:38.698472    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93a3446_61bb_4619_9c7e_96b51398ff73.slice/crio-6118787a56afa6d5cbd2979b71cc4ff00a07b5644538798b0289ef942cf7a656 WatchSource:0}: Error finding container 6118787a56afa6d5cbd2979b71cc4ff00a07b5644538798b0289ef942cf7a656: Status 404 returned error can't find the container with id 6118787a56afa6d5cbd2979b71cc4ff00a07b5644538798b0289ef942cf7a656
Jul 14 15:55:39.558956 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:39.558917    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb" event={"ID":"f93a3446-61bb-4619-9c7e-96b51398ff73","Type":"ContainerStarted","Data":"6118787a56afa6d5cbd2979b71cc4ff00a07b5644538798b0289ef942cf7a656"}
Jul 14 15:55:39.560427 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:39.560400    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv" event={"ID":"bfaff471-4b4c-4986-a88d-7b94d15f86da","Type":"ContainerStarted","Data":"d219658d8f8d632e83e082f37eb7bf2f8f78fc34e84f2d41c46f2592c7db55fa"}
Jul 14 15:55:39.582379 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:39.582327    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z72nv" podStartSLOduration=1.8607915080000001 podStartE2EDuration="25.582312367s" totalImagesPullingTime="23.721520859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:55:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:55:39.580278436 +0000 UTC m=+523.099645870" watchObservedRunningTime="2026-07-14 15:55:39.582312367 +0000 UTC m=+523.101679788"
Jul 14 15:55:43.582373 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:43.582342    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb" event={"ID":"f93a3446-61bb-4619-9c7e-96b51398ff73","Type":"ContainerStarted","Data":"cda4ad51acd53ad66cd10a8c71d64e878189d49cdf991cc25783e5ae3416e707"}
Jul 14 15:55:43.582808 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:43.582413    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb"
Jul 14 15:55:43.599085 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:43.599025    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb" podStartSLOduration=14.486982919 podStartE2EDuration="18.599009396s" totalImagesPullingTime="4.112026477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:55:43.598563086 +0000 UTC m=+527.117930532" watchObservedRunningTime="2026-07-14 15:55:43.599009396 +0000 UTC m=+527.118376817"
Jul 14 15:55:54.589432 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:55:54.589355    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nv6pb"
Jul 14 15:56:56.907948 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:56:56.907921    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 15:56:56.911166 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:56:56.911143    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 15:57:17.425778 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.425697    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"]
Jul 14 15:57:17.429365 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.429346    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"
Jul 14 15:57:17.432117 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.432096    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zptcs\""
Jul 14 15:57:17.432265 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.432096    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 14 15:57:17.439082 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.439062    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"]
Jul 14 15:57:17.472482 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.472456    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pr9p\" (UniqueName: \"kubernetes.io/projected/fe798d69-e937-4478-8208-73f94dac44f7-kube-api-access-6pr9p\") pod \"payload-pre-processing-74584bb9f8-j5frn\" (UID: \"fe798d69-e937-4478-8208-73f94dac44f7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"
Jul 14 15:57:17.472603 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.472545    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe798d69-e937-4478-8208-73f94dac44f7-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-j5frn\" (UID: \"fe798d69-e937-4478-8208-73f94dac44f7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"
Jul 14 15:57:17.574038 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.574008    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe798d69-e937-4478-8208-73f94dac44f7-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-j5frn\" (UID: \"fe798d69-e937-4478-8208-73f94dac44f7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"
Jul 14 15:57:17.574219 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.574045    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pr9p\" (UniqueName: \"kubernetes.io/projected/fe798d69-e937-4478-8208-73f94dac44f7-kube-api-access-6pr9p\") pod \"payload-pre-processing-74584bb9f8-j5frn\" (UID: \"fe798d69-e937-4478-8208-73f94dac44f7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"
Jul 14 15:57:17.574812 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.574789    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe798d69-e937-4478-8208-73f94dac44f7-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-j5frn\" (UID: \"fe798d69-e937-4478-8208-73f94dac44f7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"
Jul 14 15:57:17.581746 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.581724    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pr9p\" (UniqueName: \"kubernetes.io/projected/fe798d69-e937-4478-8208-73f94dac44f7-kube-api-access-6pr9p\") pod \"payload-pre-processing-74584bb9f8-j5frn\" (UID: \"fe798d69-e937-4478-8208-73f94dac44f7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"
Jul 14 15:57:17.741239 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.741162    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"
Jul 14 15:57:17.874580 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:17.874547    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"]
Jul 14 15:57:17.876300 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:57:17.876273    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe798d69_e937_4478_8208_73f94dac44f7.slice/crio-5a26076e9a47b259ef02c94932d04edb7145e5f66fc5868ec21d56f6846b9a22 WatchSource:0}: Error finding container 5a26076e9a47b259ef02c94932d04edb7145e5f66fc5868ec21d56f6846b9a22: Status 404 returned error can't find the container with id 5a26076e9a47b259ef02c94932d04edb7145e5f66fc5868ec21d56f6846b9a22
Jul 14 15:57:18.028673 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:18.028592    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn" event={"ID":"fe798d69-e937-4478-8208-73f94dac44f7","Type":"ContainerStarted","Data":"5a26076e9a47b259ef02c94932d04edb7145e5f66fc5868ec21d56f6846b9a22"}
Jul 14 15:57:20.041576 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:20.041544    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn" event={"ID":"fe798d69-e937-4478-8208-73f94dac44f7","Type":"ContainerStarted","Data":"406a16f59bfdbaefe0e1eab2b0077ffe53fb3f7376893b97c10f80606cd27427"}
Jul 14 15:57:20.041962 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:20.041641    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"
Jul 14 15:57:20.062068 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:20.062023    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn" podStartSLOduration=1.6993181910000001 podStartE2EDuration="3.062006968s" totalImagesPullingTime="1.362688777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:57:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:57:20.060662854 +0000 UTC m=+623.580030276" watchObservedRunningTime="2026-07-14 15:57:20.062006968 +0000 UTC m=+623.581374388"
Jul 14 15:57:31.048196 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:31.048168    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-j5frn"
Jul 14 15:57:45.590690 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.590637    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"]
Jul 14 15:57:45.594989 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.594964    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.598005 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.597979    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 14 15:57:45.598775 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.598759    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 14 15:57:45.598870 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.598849    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z2zwk\""
Jul 14 15:57:45.598928 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.598889    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 14 15:57:45.609892 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.609874    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"]
Jul 14 15:57:45.716034 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.716001    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.716178 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.716035    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.716178 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.716138    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6d62\" (UniqueName: \"kubernetes.io/projected/6062ac05-4f49-499b-b257-689e49d74d4b-kube-api-access-r6d62\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.716178 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.716170    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.716310 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.716190    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6062ac05-4f49-499b-b257-689e49d74d4b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.716310 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.716239    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.817317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.817286    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6d62\" (UniqueName: \"kubernetes.io/projected/6062ac05-4f49-499b-b257-689e49d74d4b-kube-api-access-r6d62\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.817317 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.817319    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.817561 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.817337    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6062ac05-4f49-499b-b257-689e49d74d4b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.817561 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.817377    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.817561 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.817551    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.817704 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.817579    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.817772 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.817753    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.817817 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.817798    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.817917 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.817893    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.819846 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.819824    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6062ac05-4f49-499b-b257-689e49d74d4b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.820169 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.820147    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6062ac05-4f49-499b-b257-689e49d74d4b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.838410 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.838372    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6d62\" (UniqueName: \"kubernetes.io/projected/6062ac05-4f49-499b-b257-689e49d74d4b-kube-api-access-r6d62\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7\" (UID: \"6062ac05-4f49-499b-b257-689e49d74d4b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:45.905653 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:45.905571    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:46.040648 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:46.040623    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"]
Jul 14 15:57:46.042186 ip-10-0-135-20 kubenswrapper[2560]: W0714 15:57:46.042159    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6062ac05_4f49_499b_b257_689e49d74d4b.slice/crio-fd42e4830cbe9474d413f897963bb89dd8ad84eec00c2a592c44f2083a3449f5 WatchSource:0}: Error finding container fd42e4830cbe9474d413f897963bb89dd8ad84eec00c2a592c44f2083a3449f5: Status 404 returned error can't find the container with id fd42e4830cbe9474d413f897963bb89dd8ad84eec00c2a592c44f2083a3449f5
Jul 14 15:57:46.043543 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:46.043526    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 15:57:46.160800 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:46.160729    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7" event={"ID":"6062ac05-4f49-499b-b257-689e49d74d4b","Type":"ContainerStarted","Data":"fd42e4830cbe9474d413f897963bb89dd8ad84eec00c2a592c44f2083a3449f5"}
Jul 14 15:57:48.172981 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:48.172942    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7" event={"ID":"6062ac05-4f49-499b-b257-689e49d74d4b","Type":"ContainerStarted","Data":"5bbd3d0fc7ad0343f444e0e8b2fd049197ba8667907b2fbbe3c4d685dbbeb48c"}
Jul 14 15:57:48.191912 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:48.191862    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7" podStartSLOduration=2.126156296 podStartE2EDuration="3.19184606s" totalImagesPullingTime="1.065689764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 15:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 15:57:48.190250406 +0000 UTC m=+651.709617844" watchObservedRunningTime="2026-07-14 15:57:48.19184606 +0000 UTC m=+651.711213481"
Jul 14 15:57:55.905939 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:55.905885    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:55.905939 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:55.905952    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:55.918673 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:55.918646    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 15:57:56.225080 ip-10-0-135-20 kubenswrapper[2560]: I0714 15:57:56.225001    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7"
Jul 14 16:01:14.870968 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:14.870934    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7c4db64696-l4h24"]
Jul 14 16:01:14.874727 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:14.874710    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c4db64696-l4h24"
Jul 14 16:01:14.877951 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:14.877926    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 14 16:01:14.878718 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:14.878701    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 14 16:01:14.878817 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:14.878703    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-279d7\""
Jul 14 16:01:14.888288 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:14.888264    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c4db64696-l4h24"]
Jul 14 16:01:14.922356 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:14.922330    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb9fd1e6-a308-4498-822f-2bd018f180b3-webhook-cert\") pod \"maas-controller-7c4db64696-l4h24\" (UID: \"bb9fd1e6-a308-4498-822f-2bd018f180b3\") " pod="opendatahub/maas-controller-7c4db64696-l4h24"
Jul 14 16:01:14.922516 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:14.922404    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kw2j\" (UniqueName: \"kubernetes.io/projected/bb9fd1e6-a308-4498-822f-2bd018f180b3-kube-api-access-7kw2j\") pod \"maas-controller-7c4db64696-l4h24\" (UID: \"bb9fd1e6-a308-4498-822f-2bd018f180b3\") " pod="opendatahub/maas-controller-7c4db64696-l4h24"
Jul 14 16:01:15.023398 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:15.023361    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb9fd1e6-a308-4498-822f-2bd018f180b3-webhook-cert\") pod \"maas-controller-7c4db64696-l4h24\" (UID: \"bb9fd1e6-a308-4498-822f-2bd018f180b3\") " pod="opendatahub/maas-controller-7c4db64696-l4h24"
Jul 14 16:01:15.023629 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:15.023426    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kw2j\" (UniqueName: \"kubernetes.io/projected/bb9fd1e6-a308-4498-822f-2bd018f180b3-kube-api-access-7kw2j\") pod \"maas-controller-7c4db64696-l4h24\" (UID: \"bb9fd1e6-a308-4498-822f-2bd018f180b3\") " pod="opendatahub/maas-controller-7c4db64696-l4h24"
Jul 14 16:01:15.026020 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:15.025996    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb9fd1e6-a308-4498-822f-2bd018f180b3-webhook-cert\") pod \"maas-controller-7c4db64696-l4h24\" (UID: \"bb9fd1e6-a308-4498-822f-2bd018f180b3\") " pod="opendatahub/maas-controller-7c4db64696-l4h24"
Jul 14 16:01:15.033000 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:15.032968    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kw2j\" (UniqueName: \"kubernetes.io/projected/bb9fd1e6-a308-4498-822f-2bd018f180b3-kube-api-access-7kw2j\") pod \"maas-controller-7c4db64696-l4h24\" (UID: \"bb9fd1e6-a308-4498-822f-2bd018f180b3\") " pod="opendatahub/maas-controller-7c4db64696-l4h24"
Jul 14 16:01:15.185437 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:15.185337    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c4db64696-l4h24"
Jul 14 16:01:15.539937 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:15.539912    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c4db64696-l4h24"]
Jul 14 16:01:15.542394 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:01:15.542362    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb9fd1e6_a308_4498_822f_2bd018f180b3.slice/crio-efd9cfeb6f5a36d67846fbac48605e6a1f93723014e55e9895b8edc9d5204566 WatchSource:0}: Error finding container efd9cfeb6f5a36d67846fbac48605e6a1f93723014e55e9895b8edc9d5204566: Status 404 returned error can't find the container with id efd9cfeb6f5a36d67846fbac48605e6a1f93723014e55e9895b8edc9d5204566
Jul 14 16:01:16.177926 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:16.177892    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c4db64696-l4h24" event={"ID":"bb9fd1e6-a308-4498-822f-2bd018f180b3","Type":"ContainerStarted","Data":"efd9cfeb6f5a36d67846fbac48605e6a1f93723014e55e9895b8edc9d5204566"}
Jul 14 16:01:23.219718 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:23.219670    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c4db64696-l4h24" event={"ID":"bb9fd1e6-a308-4498-822f-2bd018f180b3","Type":"ContainerStarted","Data":"b77f441ef448b88444114a00a58bb2c6e9d4be544e8b16ef49d536209db31e6b"}
Jul 14 16:01:23.220191 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:23.219869    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7c4db64696-l4h24"
Jul 14 16:01:23.253427 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:23.253380    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7c4db64696-l4h24" podStartSLOduration=2.452035259 podStartE2EDuration="9.253365696s" totalImagesPullingTime="6.801330437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:01:23.252121557 +0000 UTC m=+866.771488977" watchObservedRunningTime="2026-07-14 16:01:23.253365696 +0000 UTC m=+866.772733119"
Jul 14 16:01:34.227613 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:34.227581    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7c4db64696-l4h24"
Jul 14 16:01:56.941482 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:56.941382    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:01:56.943973 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:01:56.943949    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:06:56.974483 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:06:56.974429    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:06:56.974975 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:06:56.974576    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:11:57.004329 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:11:57.004298    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:11:57.005999 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:11:57.005977    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:13:50.359581 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.359549    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"]
Jul 14 16:13:50.362056 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.362037    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:50.364819 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.364791    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9f7be84d-serving-cert\""
Jul 14 16:13:50.365110 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.365082    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5kg5z\""
Jul 14 16:13:50.365754 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.365736    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 16:13:50.365857 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.365755    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 16:13:50.376376 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.376350    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"]
Jul 14 16:13:50.459259 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.459223    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e06fac-027e-41da-ba3d-5ebfe96618dd-maas-api-tls\") pod \"maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:50.459421 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.459262    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e06fac-027e-41da-ba3d-5ebfe96618dd-openshift-service-ca\") pod \"maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:50.459421 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.459322    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7598\" (UniqueName: \"kubernetes.io/projected/f5e06fac-027e-41da-ba3d-5ebfe96618dd-kube-api-access-s7598\") pod \"maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:50.560321 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.560286    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7598\" (UniqueName: \"kubernetes.io/projected/f5e06fac-027e-41da-ba3d-5ebfe96618dd-kube-api-access-s7598\") pod \"maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:50.560539 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.560371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e06fac-027e-41da-ba3d-5ebfe96618dd-maas-api-tls\") pod \"maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:50.560539 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.560392    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e06fac-027e-41da-ba3d-5ebfe96618dd-openshift-service-ca\") pod \"maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:50.560666 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:13:50.560556    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-serving-cert: secret "maas-api-e2e-ait-9f7be84d-serving-cert" not found
Jul 14 16:13:50.560666 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:13:50.560636    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e06fac-027e-41da-ba3d-5ebfe96618dd-maas-api-tls podName:f5e06fac-027e-41da-ba3d-5ebfe96618dd nodeName:}" failed. No retries permitted until 2026-07-14 16:13:51.060615038 +0000 UTC m=+1614.579982453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f5e06fac-027e-41da-ba3d-5ebfe96618dd-maas-api-tls") pod "maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs" (UID: "f5e06fac-027e-41da-ba3d-5ebfe96618dd") : secret "maas-api-e2e-ait-9f7be84d-serving-cert" not found
Jul 14 16:13:50.561004 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.560983    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e06fac-027e-41da-ba3d-5ebfe96618dd-openshift-service-ca\") pod \"maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:50.572060 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:50.572031    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7598\" (UniqueName: \"kubernetes.io/projected/f5e06fac-027e-41da-ba3d-5ebfe96618dd-kube-api-access-s7598\") pod \"maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:51.065437 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:51.065400    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e06fac-027e-41da-ba3d-5ebfe96618dd-maas-api-tls\") pod \"maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:51.068076 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:51.068050    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e06fac-027e-41da-ba3d-5ebfe96618dd-maas-api-tls\") pod \"maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:51.274581 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:51.274542    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:51.411855 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:51.411826    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"]
Jul 14 16:13:51.413436 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:13:51.413411    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5e06fac_027e_41da_ba3d_5ebfe96618dd.slice/crio-416347070d7f6745756c1e1e0671291c9a173eb6b1db53d4c2c09630e6bc4388 WatchSource:0}: Error finding container 416347070d7f6745756c1e1e0671291c9a173eb6b1db53d4c2c09630e6bc4388: Status 404 returned error can't find the container with id 416347070d7f6745756c1e1e0671291c9a173eb6b1db53d4c2c09630e6bc4388
Jul 14 16:13:51.414832 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:51.414812    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 16:13:51.873566 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:51.873532    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs" event={"ID":"f5e06fac-027e-41da-ba3d-5ebfe96618dd","Type":"ContainerStarted","Data":"416347070d7f6745756c1e1e0671291c9a173eb6b1db53d4c2c09630e6bc4388"}
Jul 14 16:13:53.884771 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:53.884739    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs" event={"ID":"f5e06fac-027e-41da-ba3d-5ebfe96618dd","Type":"ContainerStarted","Data":"c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7"}
Jul 14 16:13:53.885145 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:53.884847    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:13:53.901652 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:53.901588    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs" podStartSLOduration=1.548954847 podStartE2EDuration="3.901572159s" totalImagesPullingTime="2.352617312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:13:53.901166499 +0000 UTC m=+1617.420533921" watchObservedRunningTime="2026-07-14 16:13:53.901572159 +0000 UTC m=+1617.420939580"
Jul 14 16:13:59.895465 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:13:59.895409    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:14:55.722983 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:55.722948    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"]
Jul 14 16:14:55.723930 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:55.723896    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs" podUID="f5e06fac-027e-41da-ba3d-5ebfe96618dd" containerName="maas-api" containerID="cri-o://c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7" gracePeriod=30
Jul 14 16:14:55.972259 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:55.972236    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:14:55.988487 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:55.988417    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e06fac-027e-41da-ba3d-5ebfe96618dd-openshift-service-ca\") pod \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") "
Jul 14 16:14:55.988584 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:55.988521    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e06fac-027e-41da-ba3d-5ebfe96618dd-maas-api-tls\") pod \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") "
Jul 14 16:14:55.988584 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:55.988567    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7598\" (UniqueName: \"kubernetes.io/projected/f5e06fac-027e-41da-ba3d-5ebfe96618dd-kube-api-access-s7598\") pod \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\" (UID: \"f5e06fac-027e-41da-ba3d-5ebfe96618dd\") "
Jul 14 16:14:55.988735 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:55.988711    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5e06fac-027e-41da-ba3d-5ebfe96618dd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f5e06fac-027e-41da-ba3d-5ebfe96618dd" (UID: "f5e06fac-027e-41da-ba3d-5ebfe96618dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:14:55.991213 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:55.991181    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5e06fac-027e-41da-ba3d-5ebfe96618dd-kube-api-access-s7598" (OuterVolumeSpecName: "kube-api-access-s7598") pod "f5e06fac-027e-41da-ba3d-5ebfe96618dd" (UID: "f5e06fac-027e-41da-ba3d-5ebfe96618dd"). InnerVolumeSpecName "kube-api-access-s7598". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:14:55.992002 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:55.991977    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5e06fac-027e-41da-ba3d-5ebfe96618dd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f5e06fac-027e-41da-ba3d-5ebfe96618dd" (UID: "f5e06fac-027e-41da-ba3d-5ebfe96618dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:14:56.089144 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.089116    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e06fac-027e-41da-ba3d-5ebfe96618dd-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:14:56.089144 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.089141    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7598\" (UniqueName: \"kubernetes.io/projected/f5e06fac-027e-41da-ba3d-5ebfe96618dd-kube-api-access-s7598\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:14:56.089315 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.089151    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e06fac-027e-41da-ba3d-5ebfe96618dd-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:14:56.195025 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.194991    2560 generic.go:356] "Generic (PLEG): container finished" podID="f5e06fac-027e-41da-ba3d-5ebfe96618dd" containerID="c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7" exitCode=0
Jul 14 16:14:56.195174 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.195043    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"
Jul 14 16:14:56.195174 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.195079    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs" event={"ID":"f5e06fac-027e-41da-ba3d-5ebfe96618dd","Type":"ContainerDied","Data":"c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7"}
Jul 14 16:14:56.195174 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.195132    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs" event={"ID":"f5e06fac-027e-41da-ba3d-5ebfe96618dd","Type":"ContainerDied","Data":"416347070d7f6745756c1e1e0671291c9a173eb6b1db53d4c2c09630e6bc4388"}
Jul 14 16:14:56.195174 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.195154    2560 scope.go:117] "RemoveContainer" containerID="c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7"
Jul 14 16:14:56.204748 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.204733    2560 scope.go:117] "RemoveContainer" containerID="c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7"
Jul 14 16:14:56.204993 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:14:56.204972    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7\": container with ID starting with c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7 not found: ID does not exist" containerID="c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7"
Jul 14 16:14:56.205075 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.205000    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7"} err="failed to get container status \"c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7\": rpc error: code = NotFound desc = could not find container \"c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7\": container with ID starting with c85087668aae079ad7f012b629d8bb29e10b0068bfd64cb6b52e38c0378178c7 not found: ID does not exist"
Jul 14 16:14:56.217795 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.217774    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"]
Jul 14 16:14:56.222383 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:56.222360    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9f7be84d-67c7d4c6b5-bcqqs"]
Jul 14 16:14:57.033435 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:14:57.033405    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5e06fac-027e-41da-ba3d-5ebfe96618dd" path="/var/lib/kubelet/pods/f5e06fac-027e-41da-ba3d-5ebfe96618dd/volumes"
Jul 14 16:15:00.160675 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:00.160643    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj"]
Jul 14 16:15:00.161071 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:00.161057    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e06fac-027e-41da-ba3d-5ebfe96618dd" containerName="maas-api"
Jul 14 16:15:00.161114 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:00.161073    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e06fac-027e-41da-ba3d-5ebfe96618dd" containerName="maas-api"
Jul 14 16:15:00.161163 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:00.161153    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e06fac-027e-41da-ba3d-5ebfe96618dd" containerName="maas-api"
Jul 14 16:15:00.164259 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:00.164244    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj"
Jul 14 16:15:00.167921 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:00.167903    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:15:00.173926 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:00.173781    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj"
Jul 14 16:15:00.176268 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:00.176034    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj"]
Jul 14 16:15:00.313566 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:00.313544    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj"]
Jul 14 16:15:00.315182 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:15:00.315150    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294f8691_010b_4a61_b90f_6fb9fcb6481d.slice/crio-e2cfb7cd098f6e7cd0070e764511078297cbe03f635720d71c828905860df120 WatchSource:0}: Error finding container e2cfb7cd098f6e7cd0070e764511078297cbe03f635720d71c828905860df120: Status 404 returned error can't find the container with id e2cfb7cd098f6e7cd0070e764511078297cbe03f635720d71c828905860df120
Jul 14 16:15:01.220269 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:01.220232    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj" event={"ID":"294f8691-010b-4a61-b90f-6fb9fcb6481d","Type":"ContainerStarted","Data":"e2cfb7cd098f6e7cd0070e764511078297cbe03f635720d71c828905860df120"}
Jul 14 16:15:03.231252 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:03.231209    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj" event={"ID":"294f8691-010b-4a61-b90f-6fb9fcb6481d","Type":"ContainerStarted","Data":"33e040b8dc0a66dd8262233fe45500865d901fe6bb55a4d34ecefe6d7606748e"}
Jul 14 16:15:03.270393 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:03.270340    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj" podStartSLOduration=0.841201376 podStartE2EDuration="3.270323627s" totalImagesPullingTime="2.429122251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:15:03.269838367 +0000 UTC m=+1686.789205788" watchObservedRunningTime="2026-07-14 16:15:03.270323627 +0000 UTC m=+1686.789691078"
Jul 14 16:15:04.238437 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:04.238397    2560 generic.go:356] "Generic (PLEG): container finished" podID="294f8691-010b-4a61-b90f-6fb9fcb6481d" containerID="33e040b8dc0a66dd8262233fe45500865d901fe6bb55a4d34ecefe6d7606748e" exitCode=0
Jul 14 16:15:04.238837 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:04.238478    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj" event={"ID":"294f8691-010b-4a61-b90f-6fb9fcb6481d","Type":"ContainerDied","Data":"33e040b8dc0a66dd8262233fe45500865d901fe6bb55a4d34ecefe6d7606748e"}
Jul 14 16:15:05.382730 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:05.382707    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj"
Jul 14 16:15:06.251440 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:06.251411    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj"
Jul 14 16:15:06.251644 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:06.251470    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734095-djtdj" event={"ID":"294f8691-010b-4a61-b90f-6fb9fcb6481d","Type":"ContainerDied","Data":"e2cfb7cd098f6e7cd0070e764511078297cbe03f635720d71c828905860df120"}
Jul 14 16:15:06.251644 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:06.251517    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2cfb7cd098f6e7cd0070e764511078297cbe03f635720d71c828905860df120"
Jul 14 16:15:12.721363 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.721327    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"]
Jul 14 16:15:12.721843 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.721751    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="294f8691-010b-4a61-b90f-6fb9fcb6481d" containerName="cleanup"
Jul 14 16:15:12.721843 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.721762    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="294f8691-010b-4a61-b90f-6fb9fcb6481d" containerName="cleanup"
Jul 14 16:15:12.721843 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.721834    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="294f8691-010b-4a61-b90f-6fb9fcb6481d" containerName="cleanup"
Jul 14 16:15:12.725242 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.725220    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:12.727680 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.727652    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5kg5z\""
Jul 14 16:15:12.727829 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.727808    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 16:15:12.728598 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.728583    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-62755357-serving-cert\""
Jul 14 16:15:12.728689 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.728627    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 16:15:12.736382 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.736358    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"]
Jul 14 16:15:12.843147 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.843115    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-maas-api-tls\") pod \"maas-api-e2e-migrate-62755357-844d77b797-r5954\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:12.843317 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.843158    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnr7x\" (UniqueName: \"kubernetes.io/projected/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-kube-api-access-nnr7x\") pod \"maas-api-e2e-migrate-62755357-844d77b797-r5954\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:12.843317 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.843256    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-openshift-service-ca\") pod \"maas-api-e2e-migrate-62755357-844d77b797-r5954\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:12.944234 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.944198    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-maas-api-tls\") pod \"maas-api-e2e-migrate-62755357-844d77b797-r5954\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:12.944398 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.944248    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnr7x\" (UniqueName: \"kubernetes.io/projected/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-kube-api-access-nnr7x\") pod \"maas-api-e2e-migrate-62755357-844d77b797-r5954\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:12.944398 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.944318    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-openshift-service-ca\") pod \"maas-api-e2e-migrate-62755357-844d77b797-r5954\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:12.944398 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:15:12.944361    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-serving-cert: secret "maas-api-e2e-migrate-62755357-serving-cert" not found
Jul 14 16:15:12.944574 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:15:12.944422    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-maas-api-tls podName:fdb1d38f-7b66-49da-b122-d0b5a8a37e9b nodeName:}" failed. No retries permitted until 2026-07-14 16:15:13.444404854 +0000 UTC m=+1696.963772252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-maas-api-tls") pod "maas-api-e2e-migrate-62755357-844d77b797-r5954" (UID: "fdb1d38f-7b66-49da-b122-d0b5a8a37e9b") : secret "maas-api-e2e-migrate-62755357-serving-cert" not found
Jul 14 16:15:12.945039 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.945020    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-openshift-service-ca\") pod \"maas-api-e2e-migrate-62755357-844d77b797-r5954\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:12.952683 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:12.952664    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnr7x\" (UniqueName: \"kubernetes.io/projected/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-kube-api-access-nnr7x\") pod \"maas-api-e2e-migrate-62755357-844d77b797-r5954\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:13.449385 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:13.449358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-maas-api-tls\") pod \"maas-api-e2e-migrate-62755357-844d77b797-r5954\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:13.452056 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:13.452033    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-maas-api-tls\") pod \"maas-api-e2e-migrate-62755357-844d77b797-r5954\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:13.637905 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:13.637862    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:13.775144 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:13.775116    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"]
Jul 14 16:15:13.776978 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:15:13.776941    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb1d38f_7b66_49da_b122_d0b5a8a37e9b.slice/crio-926c205be4d98f9b211143eb4590f9a24d806dbc4e8512b2452f3691a4b5ae33 WatchSource:0}: Error finding container 926c205be4d98f9b211143eb4590f9a24d806dbc4e8512b2452f3691a4b5ae33: Status 404 returned error can't find the container with id 926c205be4d98f9b211143eb4590f9a24d806dbc4e8512b2452f3691a4b5ae33
Jul 14 16:15:14.292168 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:14.292132    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954" event={"ID":"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b","Type":"ContainerStarted","Data":"15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497"}
Jul 14 16:15:14.292388 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:14.292200    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:14.292388 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:14.292223    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954" event={"ID":"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b","Type":"ContainerStarted","Data":"926c205be4d98f9b211143eb4590f9a24d806dbc4e8512b2452f3691a4b5ae33"}
Jul 14 16:15:14.318591 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:14.318542    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954" podStartSLOduration=1.972274218 podStartE2EDuration="2.318525433s" totalImagesPullingTime="346.251215ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:15:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:15:14.317694067 +0000 UTC m=+1697.837061491" watchObservedRunningTime="2026-07-14 16:15:14.318525433 +0000 UTC m=+1697.837892855"
Jul 14 16:15:17.293499 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:17.293408    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"]
Jul 14 16:15:17.299698 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:17.299669    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"
Jul 14 16:15:17.304023 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:17.304002    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:15:17.311581 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:17.311560    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"]
Jul 14 16:15:17.387093 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:17.387058    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9998f666-83b6-49fd-8818-7d08c90688ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-62755357-rv8xg\" (UID: \"9998f666-83b6-49fd-8818-7d08c90688ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"
Jul 14 16:15:17.487777 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:17.487739    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9998f666-83b6-49fd-8818-7d08c90688ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-62755357-rv8xg\" (UID: \"9998f666-83b6-49fd-8818-7d08c90688ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"
Jul 14 16:15:17.488376 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:17.488357    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9998f666-83b6-49fd-8818-7d08c90688ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-62755357-rv8xg\" (UID: \"9998f666-83b6-49fd-8818-7d08c90688ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"
Jul 14 16:15:17.612555 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:17.612524    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"
Jul 14 16:15:17.756203 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:17.756180    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"]
Jul 14 16:15:17.758424 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:15:17.758392    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9998f666_83b6_49fd_8818_7d08c90688ed.slice/crio-429cb5c87614082915de392cce427d84b0d34202f3bc51f5d3b7489b9e47b223 WatchSource:0}: Error finding container 429cb5c87614082915de392cce427d84b0d34202f3bc51f5d3b7489b9e47b223: Status 404 returned error can't find the container with id 429cb5c87614082915de392cce427d84b0d34202f3bc51f5d3b7489b9e47b223
Jul 14 16:15:18.314240 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:18.314148    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg" event={"ID":"9998f666-83b6-49fd-8818-7d08c90688ed","Type":"ContainerStarted","Data":"f1efd2118b69dadddfe026a2ac79e2c5f5860733a9c1987c37252fe9e52e014e"}
Jul 14 16:15:18.314240 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:18.314212    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg" event={"ID":"9998f666-83b6-49fd-8818-7d08c90688ed","Type":"ContainerStarted","Data":"429cb5c87614082915de392cce427d84b0d34202f3bc51f5d3b7489b9e47b223"}
Jul 14 16:15:18.334352 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:18.334303    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg" podStartSLOduration=1.334288204 podStartE2EDuration="1.334288204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:15:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:15:18.331866058 +0000 UTC m=+1701.851233489" watchObservedRunningTime="2026-07-14 16:15:18.334288204 +0000 UTC m=+1701.853655666"
Jul 14 16:15:19.320257 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:19.320222    2560 generic.go:356] "Generic (PLEG): container finished" podID="9998f666-83b6-49fd-8818-7d08c90688ed" containerID="f1efd2118b69dadddfe026a2ac79e2c5f5860733a9c1987c37252fe9e52e014e" exitCode=7
Jul 14 16:15:19.320688 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:19.320310    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg" event={"ID":"9998f666-83b6-49fd-8818-7d08c90688ed","Type":"ContainerDied","Data":"f1efd2118b69dadddfe026a2ac79e2c5f5860733a9c1987c37252fe9e52e014e"}
Jul 14 16:15:19.320688 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:19.320528    2560 scope.go:117] "RemoveContainer" containerID="f1efd2118b69dadddfe026a2ac79e2c5f5860733a9c1987c37252fe9e52e014e"
Jul 14 16:15:20.302439 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:20.302362    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:15:20.329023 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:20.328986    2560 generic.go:356] "Generic (PLEG): container finished" podID="9998f666-83b6-49fd-8818-7d08c90688ed" containerID="8beb3c4553fe51fb48e164933f75d9b212d901ec7c369c0f1e931624d8d3b4e0" exitCode=7
Jul 14 16:15:20.329376 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:20.329068    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg" event={"ID":"9998f666-83b6-49fd-8818-7d08c90688ed","Type":"ContainerDied","Data":"8beb3c4553fe51fb48e164933f75d9b212d901ec7c369c0f1e931624d8d3b4e0"}
Jul 14 16:15:20.329376 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:20.329122    2560 scope.go:117] "RemoveContainer" containerID="f1efd2118b69dadddfe026a2ac79e2c5f5860733a9c1987c37252fe9e52e014e"
Jul 14 16:15:20.329376 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:20.329231    2560 scope.go:117] "RemoveContainer" containerID="8beb3c4553fe51fb48e164933f75d9b212d901ec7c369c0f1e931624d8d3b4e0"
Jul 14 16:15:20.329578 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:15:20.329439    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-62755357-rv8xg_odh-ai-gateway-infra(9998f666-83b6-49fd-8818-7d08c90688ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg" podUID="9998f666-83b6-49fd-8818-7d08c90688ed"
Jul 14 16:15:21.336052 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:21.336022    2560 scope.go:117] "RemoveContainer" containerID="8beb3c4553fe51fb48e164933f75d9b212d901ec7c369c0f1e931624d8d3b4e0"
Jul 14 16:15:21.336538 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:15:21.336270    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-62755357-rv8xg_odh-ai-gateway-infra(9998f666-83b6-49fd-8818-7d08c90688ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg" podUID="9998f666-83b6-49fd-8818-7d08c90688ed"
Jul 14 16:15:34.026208 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:34.026179    2560 scope.go:117] "RemoveContainer" containerID="8beb3c4553fe51fb48e164933f75d9b212d901ec7c369c0f1e931624d8d3b4e0"
Jul 14 16:15:34.402098 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:34.402068    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg" event={"ID":"9998f666-83b6-49fd-8818-7d08c90688ed","Type":"ContainerStarted","Data":"b3ac78693207b6beea30e36e9779e74a0ad2deb3fa520d3f98af0419cf7d65e3"}
Jul 14 16:15:35.408774 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:35.408741    2560 generic.go:356] "Generic (PLEG): container finished" podID="9998f666-83b6-49fd-8818-7d08c90688ed" containerID="b3ac78693207b6beea30e36e9779e74a0ad2deb3fa520d3f98af0419cf7d65e3" exitCode=0
Jul 14 16:15:35.409275 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:35.408788    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg" event={"ID":"9998f666-83b6-49fd-8818-7d08c90688ed","Type":"ContainerDied","Data":"b3ac78693207b6beea30e36e9779e74a0ad2deb3fa520d3f98af0419cf7d65e3"}
Jul 14 16:15:35.409275 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:35.408847    2560 scope.go:117] "RemoveContainer" containerID="8beb3c4553fe51fb48e164933f75d9b212d901ec7c369c0f1e931624d8d3b4e0"
Jul 14 16:15:35.438002 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:35.437967    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"]
Jul 14 16:15:36.550925 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:36.550898    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"
Jul 14 16:15:36.568703 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:36.568672    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9998f666-83b6-49fd-8818-7d08c90688ed-maas-api-service-ca\") pod \"9998f666-83b6-49fd-8818-7d08c90688ed\" (UID: \"9998f666-83b6-49fd-8818-7d08c90688ed\") "
Jul 14 16:15:36.569042 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:36.569018    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9998f666-83b6-49fd-8818-7d08c90688ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9998f666-83b6-49fd-8818-7d08c90688ed" (UID: "9998f666-83b6-49fd-8818-7d08c90688ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:15:36.670364 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:36.670283    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9998f666-83b6-49fd-8818-7d08c90688ed-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:15:37.421519 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:37.421491    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"
Jul 14 16:15:37.421684 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:37.421490    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg" event={"ID":"9998f666-83b6-49fd-8818-7d08c90688ed","Type":"ContainerDied","Data":"429cb5c87614082915de392cce427d84b0d34202f3bc51f5d3b7489b9e47b223"}
Jul 14 16:15:37.421684 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:37.421606    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="429cb5c87614082915de392cce427d84b0d34202f3bc51f5d3b7489b9e47b223"
Jul 14 16:15:37.468399 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:37.468373    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"]
Jul 14 16:15:37.471353 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:37.471332    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-rv8xg"]
Jul 14 16:15:39.032354 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:15:39.032319    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9998f666-83b6-49fd-8818-7d08c90688ed" path="/var/lib/kubelet/pods/9998f666-83b6-49fd-8818-7d08c90688ed/volumes"
Jul 14 16:16:17.354992 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.354954    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m"]
Jul 14 16:16:17.355616 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.355597    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9998f666-83b6-49fd-8818-7d08c90688ed" containerName="revoke-keys"
Jul 14 16:16:17.355678 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.355620    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9998f666-83b6-49fd-8818-7d08c90688ed" containerName="revoke-keys"
Jul 14 16:16:17.355678 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.355636    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9998f666-83b6-49fd-8818-7d08c90688ed" containerName="revoke-keys"
Jul 14 16:16:17.355678 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.355644    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9998f666-83b6-49fd-8818-7d08c90688ed" containerName="revoke-keys"
Jul 14 16:16:17.355678 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.355666    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9998f666-83b6-49fd-8818-7d08c90688ed" containerName="revoke-keys"
Jul 14 16:16:17.355678 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.355674    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9998f666-83b6-49fd-8818-7d08c90688ed" containerName="revoke-keys"
Jul 14 16:16:17.355831 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.355776    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9998f666-83b6-49fd-8818-7d08c90688ed" containerName="revoke-keys"
Jul 14 16:16:17.355831 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.355791    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9998f666-83b6-49fd-8818-7d08c90688ed" containerName="revoke-keys"
Jul 14 16:16:17.359534 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.359513    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m"
Jul 14 16:16:17.362210 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.362188    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:16:17.367303 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.367258    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m"]
Jul 14 16:16:17.442516 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.442486    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e1a734-77e7-461b-b373-79259865d8e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-62755357-lkl4m\" (UID: \"92e1a734-77e7-461b-b373-79259865d8e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m"
Jul 14 16:16:17.544102 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.544066    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e1a734-77e7-461b-b373-79259865d8e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-62755357-lkl4m\" (UID: \"92e1a734-77e7-461b-b373-79259865d8e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m"
Jul 14 16:16:17.544930 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.544905    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e1a734-77e7-461b-b373-79259865d8e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-62755357-lkl4m\" (UID: \"92e1a734-77e7-461b-b373-79259865d8e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m"
Jul 14 16:16:17.671570 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.671484    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m"
Jul 14 16:16:17.810076 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:17.809995    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m"]
Jul 14 16:16:17.812536 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:16:17.812510    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e1a734_77e7_461b_b373_79259865d8e9.slice/crio-5b8a03475dc348635b5343615685155883a4735d90fc0218d12c4af801d088d9 WatchSource:0}: Error finding container 5b8a03475dc348635b5343615685155883a4735d90fc0218d12c4af801d088d9: Status 404 returned error can't find the container with id 5b8a03475dc348635b5343615685155883a4735d90fc0218d12c4af801d088d9
Jul 14 16:16:18.625325 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:18.625290    2560 generic.go:356] "Generic (PLEG): container finished" podID="92e1a734-77e7-461b-b373-79259865d8e9" containerID="b720d6782b7c2d769ae5bdb696d93d45a38f40cb525ad58287bf964c197b9632" exitCode=0
Jul 14 16:16:18.625751 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:18.625387    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m" event={"ID":"92e1a734-77e7-461b-b373-79259865d8e9","Type":"ContainerDied","Data":"b720d6782b7c2d769ae5bdb696d93d45a38f40cb525ad58287bf964c197b9632"}
Jul 14 16:16:18.625751 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:18.625467    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m" event={"ID":"92e1a734-77e7-461b-b373-79259865d8e9","Type":"ContainerStarted","Data":"5b8a03475dc348635b5343615685155883a4735d90fc0218d12c4af801d088d9"}
Jul 14 16:16:19.760000 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:19.759978    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m"
Jul 14 16:16:19.866485 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:19.866426    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e1a734-77e7-461b-b373-79259865d8e9-maas-api-service-ca\") pod \"92e1a734-77e7-461b-b373-79259865d8e9\" (UID: \"92e1a734-77e7-461b-b373-79259865d8e9\") "
Jul 14 16:16:19.866766 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:19.866744    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92e1a734-77e7-461b-b373-79259865d8e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "92e1a734-77e7-461b-b373-79259865d8e9" (UID: "92e1a734-77e7-461b-b373-79259865d8e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:16:19.967199 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:19.967122    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e1a734-77e7-461b-b373-79259865d8e9-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:16:20.639218 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:20.639165    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m"
Jul 14 16:16:20.639571 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:20.639169    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-62755357-lkl4m" event={"ID":"92e1a734-77e7-461b-b373-79259865d8e9","Type":"ContainerDied","Data":"5b8a03475dc348635b5343615685155883a4735d90fc0218d12c4af801d088d9"}
Jul 14 16:16:20.639571 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:20.639331    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b8a03475dc348635b5343615685155883a4735d90fc0218d12c4af801d088d9"
Jul 14 16:16:24.212887 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.212851    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"]
Jul 14 16:16:24.213325 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.213088    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954" podUID="fdb1d38f-7b66-49da-b122-d0b5a8a37e9b" containerName="maas-api" containerID="cri-o://15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497" gracePeriod=30
Jul 14 16:16:24.462481 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.462459    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:16:24.510314 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.510241    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-openshift-service-ca\") pod \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") "
Jul 14 16:16:24.510440 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.510335    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnr7x\" (UniqueName: \"kubernetes.io/projected/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-kube-api-access-nnr7x\") pod \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") "
Jul 14 16:16:24.510440 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.510355    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-maas-api-tls\") pod \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\" (UID: \"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b\") "
Jul 14 16:16:24.510677 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.510650    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fdb1d38f-7b66-49da-b122-d0b5a8a37e9b" (UID: "fdb1d38f-7b66-49da-b122-d0b5a8a37e9b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:16:24.512545 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.512523    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fdb1d38f-7b66-49da-b122-d0b5a8a37e9b" (UID: "fdb1d38f-7b66-49da-b122-d0b5a8a37e9b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:16:24.512626 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.512576    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-kube-api-access-nnr7x" (OuterVolumeSpecName: "kube-api-access-nnr7x") pod "fdb1d38f-7b66-49da-b122-d0b5a8a37e9b" (UID: "fdb1d38f-7b66-49da-b122-d0b5a8a37e9b"). InnerVolumeSpecName "kube-api-access-nnr7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:16:24.611564 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.611529    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnr7x\" (UniqueName: \"kubernetes.io/projected/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-kube-api-access-nnr7x\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:16:24.611564 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.611557    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:16:24.611564 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.611573    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:16:24.661585 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.661553    2560 generic.go:356] "Generic (PLEG): container finished" podID="fdb1d38f-7b66-49da-b122-d0b5a8a37e9b" containerID="15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497" exitCode=0
Jul 14 16:16:24.661724 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.661598    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"
Jul 14 16:16:24.661724 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.661630    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954" event={"ID":"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b","Type":"ContainerDied","Data":"15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497"}
Jul 14 16:16:24.661724 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.661671    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954" event={"ID":"fdb1d38f-7b66-49da-b122-d0b5a8a37e9b","Type":"ContainerDied","Data":"926c205be4d98f9b211143eb4590f9a24d806dbc4e8512b2452f3691a4b5ae33"}
Jul 14 16:16:24.661724 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.661694    2560 scope.go:117] "RemoveContainer" containerID="15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497"
Jul 14 16:16:24.671201 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.671183    2560 scope.go:117] "RemoveContainer" containerID="15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497"
Jul 14 16:16:24.671485 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:16:24.671465    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497\": container with ID starting with 15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497 not found: ID does not exist" containerID="15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497"
Jul 14 16:16:24.671552 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.671492    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497"} err="failed to get container status \"15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497\": rpc error: code = NotFound desc = could not find container \"15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497\": container with ID starting with 15f5589d2ec26e37c7b53230e3a8f893b4d4128b93c06209dfa827971f970497 not found: ID does not exist"
Jul 14 16:16:24.685157 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.685136    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"]
Jul 14 16:16:24.689269 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:24.689247    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-62755357-844d77b797-r5954"]
Jul 14 16:16:25.032565 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:25.032532    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdb1d38f-7b66-49da-b122-d0b5a8a37e9b" path="/var/lib/kubelet/pods/fdb1d38f-7b66-49da-b122-d0b5a8a37e9b/volumes"
Jul 14 16:16:32.636991 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.636958    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"]
Jul 14 16:16:32.637345 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.637338    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92e1a734-77e7-461b-b373-79259865d8e9" containerName="revoke-keys"
Jul 14 16:16:32.637390 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.637348    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="92e1a734-77e7-461b-b373-79259865d8e9" containerName="revoke-keys"
Jul 14 16:16:32.637390 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.637362    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdb1d38f-7b66-49da-b122-d0b5a8a37e9b" containerName="maas-api"
Jul 14 16:16:32.637390 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.637368    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdb1d38f-7b66-49da-b122-d0b5a8a37e9b" containerName="maas-api"
Jul 14 16:16:32.637513 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.637438    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="92e1a734-77e7-461b-b373-79259865d8e9" containerName="revoke-keys"
Jul 14 16:16:32.637513 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.637463    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdb1d38f-7b66-49da-b122-d0b5a8a37e9b" containerName="maas-api"
Jul 14 16:16:32.637513 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.637468    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9998f666-83b6-49fd-8818-7d08c90688ed" containerName="revoke-keys"
Jul 14 16:16:32.640288 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.640271    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:32.652282 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.652265    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7254e6a1-serving-cert\""
Jul 14 16:16:32.652377 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.652302    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 16:16:32.653323 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.653307    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5kg5z\""
Jul 14 16:16:32.653461 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.653427    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 16:16:32.666773 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.666753    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"]
Jul 14 16:16:32.780797 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.780762    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxw4g\" (UniqueName: \"kubernetes.io/projected/358af053-7a92-4415-bb15-71efbd4186c6-kube-api-access-hxw4g\") pod \"maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:32.780988 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.780821    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/358af053-7a92-4415-bb15-71efbd4186c6-openshift-service-ca\") pod \"maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:32.780988 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.780893    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/358af053-7a92-4415-bb15-71efbd4186c6-maas-api-tls\") pod \"maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:32.881507 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.881471    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/358af053-7a92-4415-bb15-71efbd4186c6-openshift-service-ca\") pod \"maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:32.881663 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.881546    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/358af053-7a92-4415-bb15-71efbd4186c6-maas-api-tls\") pod \"maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:32.881663 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.881613    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxw4g\" (UniqueName: \"kubernetes.io/projected/358af053-7a92-4415-bb15-71efbd4186c6-kube-api-access-hxw4g\") pod \"maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:32.881736 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:16:32.881693    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-serving-cert: secret "maas-api-e2e-ait-7254e6a1-serving-cert" not found
Jul 14 16:16:32.881797 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:16:32.881784    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/358af053-7a92-4415-bb15-71efbd4186c6-maas-api-tls podName:358af053-7a92-4415-bb15-71efbd4186c6 nodeName:}" failed. No retries permitted until 2026-07-14 16:16:33.381760739 +0000 UTC m=+1776.901128139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/358af053-7a92-4415-bb15-71efbd4186c6-maas-api-tls") pod "maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm" (UID: "358af053-7a92-4415-bb15-71efbd4186c6") : secret "maas-api-e2e-ait-7254e6a1-serving-cert" not found
Jul 14 16:16:32.882213 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.882196    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/358af053-7a92-4415-bb15-71efbd4186c6-openshift-service-ca\") pod \"maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:32.895918 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:32.895856    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxw4g\" (UniqueName: \"kubernetes.io/projected/358af053-7a92-4415-bb15-71efbd4186c6-kube-api-access-hxw4g\") pod \"maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:33.386012 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:33.385971    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/358af053-7a92-4415-bb15-71efbd4186c6-maas-api-tls\") pod \"maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:33.388607 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:33.388587    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/358af053-7a92-4415-bb15-71efbd4186c6-maas-api-tls\") pod \"maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:33.550636 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:33.550595    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:33.700093 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:33.700067    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"]
Jul 14 16:16:33.702484 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:16:33.702432    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358af053_7a92_4415_bb15_71efbd4186c6.slice/crio-51685aaae5ac079633b14563f049061d8e028ddffb53fbdbec0d3df61e939c80 WatchSource:0}: Error finding container 51685aaae5ac079633b14563f049061d8e028ddffb53fbdbec0d3df61e939c80: Status 404 returned error can't find the container with id 51685aaae5ac079633b14563f049061d8e028ddffb53fbdbec0d3df61e939c80
Jul 14 16:16:33.709654 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:33.709624    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm" event={"ID":"358af053-7a92-4415-bb15-71efbd4186c6","Type":"ContainerStarted","Data":"51685aaae5ac079633b14563f049061d8e028ddffb53fbdbec0d3df61e939c80"}
Jul 14 16:16:34.716199 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:34.716154    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm" event={"ID":"358af053-7a92-4415-bb15-71efbd4186c6","Type":"ContainerStarted","Data":"1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1"}
Jul 14 16:16:34.716669 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:34.716240    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:34.783247 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:34.783194    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm" podStartSLOduration=2.243835318 podStartE2EDuration="2.783179687s" totalImagesPullingTime="539.344369ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:16:34.78080458 +0000 UTC m=+1778.300172005" watchObservedRunningTime="2026-07-14 16:16:34.783179687 +0000 UTC m=+1778.302547163"
Jul 14 16:16:37.575937 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:37.575896    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"]
Jul 14 16:16:37.579932 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:37.579908    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"
Jul 14 16:16:37.583047 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:37.583027    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:16:37.589059 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:37.589033    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"]
Jul 14 16:16:37.726817 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:37.726770    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d5eeb8-58f0-43de-a056-0a5f17a0fe66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp\" (UID: \"47d5eeb8-58f0-43de-a056-0a5f17a0fe66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"
Jul 14 16:16:37.828014 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:37.827942    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d5eeb8-58f0-43de-a056-0a5f17a0fe66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp\" (UID: \"47d5eeb8-58f0-43de-a056-0a5f17a0fe66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"
Jul 14 16:16:37.828599 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:37.828577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d5eeb8-58f0-43de-a056-0a5f17a0fe66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp\" (UID: \"47d5eeb8-58f0-43de-a056-0a5f17a0fe66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"
Jul 14 16:16:37.891721 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:37.891699    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"
Jul 14 16:16:38.032911 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:38.032884    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"]
Jul 14 16:16:38.035079 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:16:38.035042    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d5eeb8_58f0_43de_a056_0a5f17a0fe66.slice/crio-4d3905274225c5297465ae5d69dcb572c95ae0047215af6731397e7f7c015737 WatchSource:0}: Error finding container 4d3905274225c5297465ae5d69dcb572c95ae0047215af6731397e7f7c015737: Status 404 returned error can't find the container with id 4d3905274225c5297465ae5d69dcb572c95ae0047215af6731397e7f7c015737
Jul 14 16:16:38.737132 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:38.737096    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp" event={"ID":"47d5eeb8-58f0-43de-a056-0a5f17a0fe66","Type":"ContainerStarted","Data":"cf595b36cdf7105bcf4fd2bf54a4d274092d735828b4df7f7fba1a2ee6d2f718"}
Jul 14 16:16:38.737538 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:38.737149    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp" event={"ID":"47d5eeb8-58f0-43de-a056-0a5f17a0fe66","Type":"ContainerStarted","Data":"4d3905274225c5297465ae5d69dcb572c95ae0047215af6731397e7f7c015737"}
Jul 14 16:16:38.755369 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:38.755327    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp" podStartSLOduration=1.755313456 podStartE2EDuration="1.755313456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:16:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:16:38.754043241 +0000 UTC m=+1782.273410662" watchObservedRunningTime="2026-07-14 16:16:38.755313456 +0000 UTC m=+1782.274680876"
Jul 14 16:16:39.743198 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:39.743156    2560 generic.go:356] "Generic (PLEG): container finished" podID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerID="cf595b36cdf7105bcf4fd2bf54a4d274092d735828b4df7f7fba1a2ee6d2f718" exitCode=7
Jul 14 16:16:39.743604 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:39.743240    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp" event={"ID":"47d5eeb8-58f0-43de-a056-0a5f17a0fe66","Type":"ContainerDied","Data":"cf595b36cdf7105bcf4fd2bf54a4d274092d735828b4df7f7fba1a2ee6d2f718"}
Jul 14 16:16:39.743604 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:39.743437    2560 scope.go:117] "RemoveContainer" containerID="cf595b36cdf7105bcf4fd2bf54a4d274092d735828b4df7f7fba1a2ee6d2f718"
Jul 14 16:16:40.728121 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:40.728090    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:16:40.750563 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:40.750529    2560 generic.go:356] "Generic (PLEG): container finished" podID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerID="86a5ceb3edcf1ece83526438ad895add5c9aff47ac9a97aaffcab57be607263b" exitCode=7
Jul 14 16:16:40.751016 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:40.750581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp" event={"ID":"47d5eeb8-58f0-43de-a056-0a5f17a0fe66","Type":"ContainerDied","Data":"86a5ceb3edcf1ece83526438ad895add5c9aff47ac9a97aaffcab57be607263b"}
Jul 14 16:16:40.751016 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:40.750666    2560 scope.go:117] "RemoveContainer" containerID="cf595b36cdf7105bcf4fd2bf54a4d274092d735828b4df7f7fba1a2ee6d2f718"
Jul 14 16:16:40.751016 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:40.750828    2560 scope.go:117] "RemoveContainer" containerID="86a5ceb3edcf1ece83526438ad895add5c9aff47ac9a97aaffcab57be607263b"
Jul 14 16:16:40.751130 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:16:40.751078    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp_odh-ai-gateway-infra(47d5eeb8-58f0-43de-a056-0a5f17a0fe66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66"
Jul 14 16:16:41.757177 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:41.757149    2560 scope.go:117] "RemoveContainer" containerID="86a5ceb3edcf1ece83526438ad895add5c9aff47ac9a97aaffcab57be607263b"
Jul 14 16:16:41.757639 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:16:41.757358    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp_odh-ai-gateway-infra(47d5eeb8-58f0-43de-a056-0a5f17a0fe66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66"
Jul 14 16:16:55.026302 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:55.026264    2560 scope.go:117] "RemoveContainer" containerID="86a5ceb3edcf1ece83526438ad895add5c9aff47ac9a97aaffcab57be607263b"
Jul 14 16:16:55.834631 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:55.834599    2560 generic.go:356] "Generic (PLEG): container finished" podID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerID="e869ca8db1528dc1a682d261abf3972f2f1916f3b4f2f0a31dbff612b0caea50" exitCode=0
Jul 14 16:16:55.834797 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:55.834674    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp" event={"ID":"47d5eeb8-58f0-43de-a056-0a5f17a0fe66","Type":"ContainerDied","Data":"e869ca8db1528dc1a682d261abf3972f2f1916f3b4f2f0a31dbff612b0caea50"}
Jul 14 16:16:55.834797 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:55.834733    2560 scope.go:117] "RemoveContainer" containerID="86a5ceb3edcf1ece83526438ad895add5c9aff47ac9a97aaffcab57be607263b"
Jul 14 16:16:56.869238 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:56.869204    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"]
Jul 14 16:16:56.978729 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:56.978703    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"
Jul 14 16:16:57.007744 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:57.007720    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d5eeb8-58f0-43de-a056-0a5f17a0fe66-maas-api-service-ca\") pod \"47d5eeb8-58f0-43de-a056-0a5f17a0fe66\" (UID: \"47d5eeb8-58f0-43de-a056-0a5f17a0fe66\") "
Jul 14 16:16:57.008049 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:57.008027    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47d5eeb8-58f0-43de-a056-0a5f17a0fe66-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "47d5eeb8-58f0-43de-a056-0a5f17a0fe66" (UID: "47d5eeb8-58f0-43de-a056-0a5f17a0fe66"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:16:57.040863 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:57.040838    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:16:57.044006 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:57.043988    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:16:57.109367 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:57.109338    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d5eeb8-58f0-43de-a056-0a5f17a0fe66-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:16:57.847777 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:57.847749    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"
Jul 14 16:16:57.847952 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:57.847793    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp" event={"ID":"47d5eeb8-58f0-43de-a056-0a5f17a0fe66","Type":"ContainerDied","Data":"4d3905274225c5297465ae5d69dcb572c95ae0047215af6731397e7f7c015737"}
Jul 14 16:16:57.847952 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:57.847876    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d3905274225c5297465ae5d69dcb572c95ae0047215af6731397e7f7c015737"
Jul 14 16:16:57.880561 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:57.880530    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"]
Jul 14 16:16:57.883018 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:57.882995    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-24cbp"]
Jul 14 16:16:59.032384 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:16:59.032350    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" path="/var/lib/kubelet/pods/47d5eeb8-58f0-43de-a056-0a5f17a0fe66/volumes"
Jul 14 16:17:37.633882 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.633848    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z"]
Jul 14 16:17:37.634404 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.634255    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerName="revoke-keys"
Jul 14 16:17:37.634404 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.634266    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerName="revoke-keys"
Jul 14 16:17:37.634404 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.634274    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerName="revoke-keys"
Jul 14 16:17:37.634404 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.634279    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerName="revoke-keys"
Jul 14 16:17:37.634404 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.634293    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerName="revoke-keys"
Jul 14 16:17:37.634404 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.634300    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerName="revoke-keys"
Jul 14 16:17:37.634404 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.634381    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerName="revoke-keys"
Jul 14 16:17:37.634404 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.634388    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerName="revoke-keys"
Jul 14 16:17:37.634404 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.634394    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="47d5eeb8-58f0-43de-a056-0a5f17a0fe66" containerName="revoke-keys"
Jul 14 16:17:37.637574 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.637558    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z"
Jul 14 16:17:37.639736 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.639715    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:17:37.645187 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.645162    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z"]
Jul 14 16:17:37.677841 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.677809    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb65506-1da9-460b-b90b-926ad28a81de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z\" (UID: \"bcb65506-1da9-460b-b90b-926ad28a81de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z"
Jul 14 16:17:37.779301 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.779271    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb65506-1da9-460b-b90b-926ad28a81de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z\" (UID: \"bcb65506-1da9-460b-b90b-926ad28a81de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z"
Jul 14 16:17:37.779913 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.779894    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb65506-1da9-460b-b90b-926ad28a81de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z\" (UID: \"bcb65506-1da9-460b-b90b-926ad28a81de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z"
Jul 14 16:17:37.949925 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:37.949847    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z"
Jul 14 16:17:38.086845 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:38.086818    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z"]
Jul 14 16:17:38.088903 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:17:38.088879    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb65506_1da9_460b_b90b_926ad28a81de.slice/crio-36c653786913d2b77655e969d84d14a2e66635b9005f54b74f8111c45166fc39 WatchSource:0}: Error finding container 36c653786913d2b77655e969d84d14a2e66635b9005f54b74f8111c45166fc39: Status 404 returned error can't find the container with id 36c653786913d2b77655e969d84d14a2e66635b9005f54b74f8111c45166fc39
Jul 14 16:17:39.055582 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:39.055487    2560 generic.go:356] "Generic (PLEG): container finished" podID="bcb65506-1da9-460b-b90b-926ad28a81de" containerID="5a669e0b2c10b590da623839c5fbb263a9053d50dbd04d6039ae652634168b55" exitCode=0
Jul 14 16:17:39.055582 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:39.055534    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z" event={"ID":"bcb65506-1da9-460b-b90b-926ad28a81de","Type":"ContainerDied","Data":"5a669e0b2c10b590da623839c5fbb263a9053d50dbd04d6039ae652634168b55"}
Jul 14 16:17:39.056005 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:39.055595    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z" event={"ID":"bcb65506-1da9-460b-b90b-926ad28a81de","Type":"ContainerStarted","Data":"36c653786913d2b77655e969d84d14a2e66635b9005f54b74f8111c45166fc39"}
Jul 14 16:17:40.195163 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:40.195132    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z"
Jul 14 16:17:40.300598 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:40.300566    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb65506-1da9-460b-b90b-926ad28a81de-maas-api-service-ca\") pod \"bcb65506-1da9-460b-b90b-926ad28a81de\" (UID: \"bcb65506-1da9-460b-b90b-926ad28a81de\") "
Jul 14 16:17:40.300947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:40.300927    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb65506-1da9-460b-b90b-926ad28a81de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bcb65506-1da9-460b-b90b-926ad28a81de" (UID: "bcb65506-1da9-460b-b90b-926ad28a81de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:17:40.401611 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:40.401581    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb65506-1da9-460b-b90b-926ad28a81de-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:17:41.067239 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:41.067211    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z"
Jul 14 16:17:41.067392 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:41.067241    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z" event={"ID":"bcb65506-1da9-460b-b90b-926ad28a81de","Type":"ContainerDied","Data":"36c653786913d2b77655e969d84d14a2e66635b9005f54b74f8111c45166fc39"}
Jul 14 16:17:41.067392 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:41.067295    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36c653786913d2b77655e969d84d14a2e66635b9005f54b74f8111c45166fc39"
Jul 14 16:17:48.288001 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.287972    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"]
Jul 14 16:17:48.288740 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.288169    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm" podUID="358af053-7a92-4415-bb15-71efbd4186c6" containerName="maas-api" containerID="cri-o://1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1" gracePeriod=30
Jul 14 16:17:48.541244 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.541184    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:17:48.573882 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.573855    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxw4g\" (UniqueName: \"kubernetes.io/projected/358af053-7a92-4415-bb15-71efbd4186c6-kube-api-access-hxw4g\") pod \"358af053-7a92-4415-bb15-71efbd4186c6\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") "
Jul 14 16:17:48.574032 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.573913    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/358af053-7a92-4415-bb15-71efbd4186c6-maas-api-tls\") pod \"358af053-7a92-4415-bb15-71efbd4186c6\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") "
Jul 14 16:17:48.574032 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.573974    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/358af053-7a92-4415-bb15-71efbd4186c6-openshift-service-ca\") pod \"358af053-7a92-4415-bb15-71efbd4186c6\" (UID: \"358af053-7a92-4415-bb15-71efbd4186c6\") "
Jul 14 16:17:48.574387 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.574359    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/358af053-7a92-4415-bb15-71efbd4186c6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "358af053-7a92-4415-bb15-71efbd4186c6" (UID: "358af053-7a92-4415-bb15-71efbd4186c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:17:48.576198 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.576177    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/358af053-7a92-4415-bb15-71efbd4186c6-kube-api-access-hxw4g" (OuterVolumeSpecName: "kube-api-access-hxw4g") pod "358af053-7a92-4415-bb15-71efbd4186c6" (UID: "358af053-7a92-4415-bb15-71efbd4186c6"). InnerVolumeSpecName "kube-api-access-hxw4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:17:48.576284 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.576215    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/358af053-7a92-4415-bb15-71efbd4186c6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "358af053-7a92-4415-bb15-71efbd4186c6" (UID: "358af053-7a92-4415-bb15-71efbd4186c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:17:48.674927 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.674894    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxw4g\" (UniqueName: \"kubernetes.io/projected/358af053-7a92-4415-bb15-71efbd4186c6-kube-api-access-hxw4g\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:17:48.674927 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.674922    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/358af053-7a92-4415-bb15-71efbd4186c6-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:17:48.674927 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:48.674933    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/358af053-7a92-4415-bb15-71efbd4186c6-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:17:49.111576 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:49.111536    2560 generic.go:356] "Generic (PLEG): container finished" podID="358af053-7a92-4415-bb15-71efbd4186c6" containerID="1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1" exitCode=0
Jul 14 16:17:49.111737 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:49.111581    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"
Jul 14 16:17:49.111737 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:49.111593    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm" event={"ID":"358af053-7a92-4415-bb15-71efbd4186c6","Type":"ContainerDied","Data":"1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1"}
Jul 14 16:17:49.111737 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:49.111642    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm" event={"ID":"358af053-7a92-4415-bb15-71efbd4186c6","Type":"ContainerDied","Data":"51685aaae5ac079633b14563f049061d8e028ddffb53fbdbec0d3df61e939c80"}
Jul 14 16:17:49.111737 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:49.111666    2560 scope.go:117] "RemoveContainer" containerID="1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1"
Jul 14 16:17:49.120345 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:49.120318    2560 scope.go:117] "RemoveContainer" containerID="1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1"
Jul 14 16:17:49.120596 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:17:49.120579    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1\": container with ID starting with 1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1 not found: ID does not exist" containerID="1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1"
Jul 14 16:17:49.120671 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:49.120608    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1"} err="failed to get container status \"1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1\": rpc error: code = NotFound desc = could not find container \"1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1\": container with ID starting with 1d399fdcce247dba8952c83406bebc0dc64fa3ba75b755b2388b9111a2aaa6e1 not found: ID does not exist"
Jul 14 16:17:49.133365 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:49.133338    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"]
Jul 14 16:17:49.142255 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:49.142228    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7254e6a1-6d6b667bff-w9sdm"]
Jul 14 16:17:51.032578 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:17:51.032545    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="358af053-7a92-4415-bb15-71efbd4186c6" path="/var/lib/kubelet/pods/358af053-7a92-4415-bb15-71efbd4186c6/volumes"
Jul 14 16:18:05.606211 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.606178    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"]
Jul 14 16:18:05.607917 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.607889    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb65506-1da9-460b-b90b-926ad28a81de" containerName="revoke-keys"
Jul 14 16:18:05.608080 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.608065    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb65506-1da9-460b-b90b-926ad28a81de" containerName="revoke-keys"
Jul 14 16:18:05.608230 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.608217    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="358af053-7a92-4415-bb15-71efbd4186c6" containerName="maas-api"
Jul 14 16:18:05.608320 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.608309    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="358af053-7a92-4415-bb15-71efbd4186c6" containerName="maas-api"
Jul 14 16:18:05.609380 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.608626    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="358af053-7a92-4415-bb15-71efbd4186c6" containerName="maas-api"
Jul 14 16:18:05.609560 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.609544    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb65506-1da9-460b-b90b-926ad28a81de" containerName="revoke-keys"
Jul 14 16:18:05.615421 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.615398    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:05.619073 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.619054    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 16:18:05.619176 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.619076    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-b8cdb4a9-serving-cert\""
Jul 14 16:18:05.619328 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.619255    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5kg5z\""
Jul 14 16:18:05.619328 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.619150    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 16:18:05.621039 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.621018    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"]
Jul 14 16:18:05.728567 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.728520    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92b97c0-408d-4571-97eb-b17d24187f3e-openshift-service-ca\") pod \"maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:05.728768 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.728594    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f92b97c0-408d-4571-97eb-b17d24187f3e-maas-api-tls\") pod \"maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:05.728768 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.728656    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8b2n\" (UniqueName: \"kubernetes.io/projected/f92b97c0-408d-4571-97eb-b17d24187f3e-kube-api-access-n8b2n\") pod \"maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:05.829734 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.829695    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92b97c0-408d-4571-97eb-b17d24187f3e-openshift-service-ca\") pod \"maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:05.829930 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.829760    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f92b97c0-408d-4571-97eb-b17d24187f3e-maas-api-tls\") pod \"maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:05.829930 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.829798    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8b2n\" (UniqueName: \"kubernetes.io/projected/f92b97c0-408d-4571-97eb-b17d24187f3e-kube-api-access-n8b2n\") pod \"maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:05.830401 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.830373    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92b97c0-408d-4571-97eb-b17d24187f3e-openshift-service-ca\") pod \"maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:05.832512 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.832487    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f92b97c0-408d-4571-97eb-b17d24187f3e-maas-api-tls\") pod \"maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:05.849617 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.849585    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8b2n\" (UniqueName: \"kubernetes.io/projected/f92b97c0-408d-4571-97eb-b17d24187f3e-kube-api-access-n8b2n\") pod \"maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:05.927647 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:05.927567    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:06.074400 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:06.074371    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"]
Jul 14 16:18:06.076218 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:18:06.076191    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf92b97c0_408d_4571_97eb_b17d24187f3e.slice/crio-97ff90c685a4e33954e87bcd0c9164a5270ac38c1b8b7120d4c30747d905da3a WatchSource:0}: Error finding container 97ff90c685a4e33954e87bcd0c9164a5270ac38c1b8b7120d4c30747d905da3a: Status 404 returned error can't find the container with id 97ff90c685a4e33954e87bcd0c9164a5270ac38c1b8b7120d4c30747d905da3a
Jul 14 16:18:06.198534 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:06.198428    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp" event={"ID":"f92b97c0-408d-4571-97eb-b17d24187f3e","Type":"ContainerStarted","Data":"97ff90c685a4e33954e87bcd0c9164a5270ac38c1b8b7120d4c30747d905da3a"}
Jul 14 16:18:07.208099 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:07.208061    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp" event={"ID":"f92b97c0-408d-4571-97eb-b17d24187f3e","Type":"ContainerStarted","Data":"d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25"}
Jul 14 16:18:07.208576 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:07.208238    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:07.229493 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:07.229426    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp" podStartSLOduration=1.828516403 podStartE2EDuration="2.229412484s" totalImagesPullingTime="400.896081ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:18:07.226912985 +0000 UTC m=+1870.746280419" watchObservedRunningTime="2026-07-14 16:18:07.229412484 +0000 UTC m=+1870.748779899"
Jul 14 16:18:09.891786 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:09.891742    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"]
Jul 14 16:18:09.895484 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:09.895459    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"
Jul 14 16:18:09.898097 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:09.898071    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:18:09.907053 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:09.907029    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"]
Jul 14 16:18:09.969056 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:09.969024    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13a13abd-aa4f-4b22-aa31-d05edb365c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r\" (UID: \"13a13abd-aa4f-4b22-aa31-d05edb365c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"
Jul 14 16:18:10.069745 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:10.069710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13a13abd-aa4f-4b22-aa31-d05edb365c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r\" (UID: \"13a13abd-aa4f-4b22-aa31-d05edb365c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"
Jul 14 16:18:10.070339 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:10.070320    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13a13abd-aa4f-4b22-aa31-d05edb365c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r\" (UID: \"13a13abd-aa4f-4b22-aa31-d05edb365c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"
Jul 14 16:18:10.208208 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:10.208133    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"
Jul 14 16:18:10.346728 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:10.346700    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"]
Jul 14 16:18:10.348440 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:18:10.348406    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13a13abd_aa4f_4b22_aa31_d05edb365c9c.slice/crio-4d5235084297a2e35fe82c901d619952df2cb0a218f076bd8598d63a550358b6 WatchSource:0}: Error finding container 4d5235084297a2e35fe82c901d619952df2cb0a218f076bd8598d63a550358b6: Status 404 returned error can't find the container with id 4d5235084297a2e35fe82c901d619952df2cb0a218f076bd8598d63a550358b6
Jul 14 16:18:11.239843 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:11.239749    2560 generic.go:356] "Generic (PLEG): container finished" podID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerID="af7a1fbf2a4de5f5b711c3e6a18e775b5a76af4329204e4a73fd8d9312efaab0" exitCode=7
Jul 14 16:18:11.239843 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:11.239811    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r" event={"ID":"13a13abd-aa4f-4b22-aa31-d05edb365c9c","Type":"ContainerDied","Data":"af7a1fbf2a4de5f5b711c3e6a18e775b5a76af4329204e4a73fd8d9312efaab0"}
Jul 14 16:18:11.240235 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:11.239848    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r" event={"ID":"13a13abd-aa4f-4b22-aa31-d05edb365c9c","Type":"ContainerStarted","Data":"4d5235084297a2e35fe82c901d619952df2cb0a218f076bd8598d63a550358b6"}
Jul 14 16:18:11.240235 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:11.239992    2560 scope.go:117] "RemoveContainer" containerID="af7a1fbf2a4de5f5b711c3e6a18e775b5a76af4329204e4a73fd8d9312efaab0"
Jul 14 16:18:12.247099 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:12.247006    2560 generic.go:356] "Generic (PLEG): container finished" podID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerID="e6607f31b4c383a763f3c2ca04c1b5f6010942797bff778d401a3516568d36e4" exitCode=7
Jul 14 16:18:12.247099 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:12.247091    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r" event={"ID":"13a13abd-aa4f-4b22-aa31-d05edb365c9c","Type":"ContainerDied","Data":"e6607f31b4c383a763f3c2ca04c1b5f6010942797bff778d401a3516568d36e4"}
Jul 14 16:18:12.247575 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:12.247145    2560 scope.go:117] "RemoveContainer" containerID="af7a1fbf2a4de5f5b711c3e6a18e775b5a76af4329204e4a73fd8d9312efaab0"
Jul 14 16:18:12.247575 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:12.247378    2560 scope.go:117] "RemoveContainer" containerID="e6607f31b4c383a763f3c2ca04c1b5f6010942797bff778d401a3516568d36e4"
Jul 14 16:18:12.247656 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:18:12.247626    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r_odh-ai-gateway-infra(13a13abd-aa4f-4b22-aa31-d05edb365c9c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c"
Jul 14 16:18:13.220028 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:13.219998    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:18:13.255112 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:13.255088    2560 scope.go:117] "RemoveContainer" containerID="e6607f31b4c383a763f3c2ca04c1b5f6010942797bff778d401a3516568d36e4"
Jul 14 16:18:13.255532 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:18:13.255306    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r_odh-ai-gateway-infra(13a13abd-aa4f-4b22-aa31-d05edb365c9c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c"
Jul 14 16:18:27.029025 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:27.028994    2560 scope.go:117] "RemoveContainer" containerID="e6607f31b4c383a763f3c2ca04c1b5f6010942797bff778d401a3516568d36e4"
Jul 14 16:18:27.329612 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:27.329581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r" event={"ID":"13a13abd-aa4f-4b22-aa31-d05edb365c9c","Type":"ContainerStarted","Data":"5653bd4bdacd58613bd60f4fb8950e1b97daa25bdd5b522f11c60e064e9d2c23"}
Jul 14 16:18:27.347228 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:27.347181    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r" podStartSLOduration=18.34716768 podStartE2EDuration="18.34716768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:18:27.345498793 +0000 UTC m=+1890.864866215" watchObservedRunningTime="2026-07-14 16:18:27.34716768 +0000 UTC m=+1890.866535101"
Jul 14 16:18:28.335942 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:28.335909    2560 generic.go:356] "Generic (PLEG): container finished" podID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerID="5653bd4bdacd58613bd60f4fb8950e1b97daa25bdd5b522f11c60e064e9d2c23" exitCode=0
Jul 14 16:18:28.336435 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:28.335989    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r" event={"ID":"13a13abd-aa4f-4b22-aa31-d05edb365c9c","Type":"ContainerDied","Data":"5653bd4bdacd58613bd60f4fb8950e1b97daa25bdd5b522f11c60e064e9d2c23"}
Jul 14 16:18:28.336435 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:28.336053    2560 scope.go:117] "RemoveContainer" containerID="e6607f31b4c383a763f3c2ca04c1b5f6010942797bff778d401a3516568d36e4"
Jul 14 16:18:28.368240 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:28.368215    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"]
Jul 14 16:18:29.475866 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:29.475839    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"
Jul 14 16:18:29.542429 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:29.542404    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13a13abd-aa4f-4b22-aa31-d05edb365c9c-maas-api-service-ca\") pod \"13a13abd-aa4f-4b22-aa31-d05edb365c9c\" (UID: \"13a13abd-aa4f-4b22-aa31-d05edb365c9c\") "
Jul 14 16:18:29.542744 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:29.542723    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13a13abd-aa4f-4b22-aa31-d05edb365c9c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "13a13abd-aa4f-4b22-aa31-d05edb365c9c" (UID: "13a13abd-aa4f-4b22-aa31-d05edb365c9c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:18:29.644048 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:29.643975    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13a13abd-aa4f-4b22-aa31-d05edb365c9c-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:18:30.349825 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:30.349792    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"
Jul 14 16:18:30.350016 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:30.349829    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r" event={"ID":"13a13abd-aa4f-4b22-aa31-d05edb365c9c","Type":"ContainerDied","Data":"4d5235084297a2e35fe82c901d619952df2cb0a218f076bd8598d63a550358b6"}
Jul 14 16:18:30.350016 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:30.349875    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d5235084297a2e35fe82c901d619952df2cb0a218f076bd8598d63a550358b6"
Jul 14 16:18:30.386256 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:30.386227    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"]
Jul 14 16:18:30.398248 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:30.394634    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-zcw8r"]
Jul 14 16:18:31.032364 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:18:31.032327    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" path="/var/lib/kubelet/pods/13a13abd-aa4f-4b22-aa31-d05edb365c9c/volumes"
Jul 14 16:19:09.947823 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:09.947786    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb"]
Jul 14 16:19:09.948243 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:09.948177    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerName="revoke-keys"
Jul 14 16:19:09.948243 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:09.948187    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerName="revoke-keys"
Jul 14 16:19:09.948243 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:09.948214    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerName="revoke-keys"
Jul 14 16:19:09.948243 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:09.948220    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerName="revoke-keys"
Jul 14 16:19:09.948375 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:09.948279    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerName="revoke-keys"
Jul 14 16:19:09.948375 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:09.948288    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerName="revoke-keys"
Jul 14 16:19:09.948375 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:09.948295    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerName="revoke-keys"
Jul 14 16:19:09.951563 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:09.951546    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb"
Jul 14 16:19:09.953960 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:09.953939    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:19:09.959167 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:09.959141    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb"]
Jul 14 16:19:10.016373 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:10.016343    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f759301-008b-4906-8f05-70329aac89ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb\" (UID: \"4f759301-008b-4906-8f05-70329aac89ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb"
Jul 14 16:19:10.117930 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:10.117895    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f759301-008b-4906-8f05-70329aac89ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb\" (UID: \"4f759301-008b-4906-8f05-70329aac89ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb"
Jul 14 16:19:10.118567 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:10.118549    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f759301-008b-4906-8f05-70329aac89ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb\" (UID: \"4f759301-008b-4906-8f05-70329aac89ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb"
Jul 14 16:19:10.264105 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:10.264019    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb"
Jul 14 16:19:10.412864 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:10.412841    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb"]
Jul 14 16:19:10.415062 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:19:10.415035    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f759301_008b_4906_8f05_70329aac89ab.slice/crio-0ea22c9b14bd58d52ee73a1599076abe221a31039cc94e2759a06a6732cbb0fc WatchSource:0}: Error finding container 0ea22c9b14bd58d52ee73a1599076abe221a31039cc94e2759a06a6732cbb0fc: Status 404 returned error can't find the container with id 0ea22c9b14bd58d52ee73a1599076abe221a31039cc94e2759a06a6732cbb0fc
Jul 14 16:19:10.558315 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:10.558233    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb" event={"ID":"4f759301-008b-4906-8f05-70329aac89ab","Type":"ContainerStarted","Data":"0ea22c9b14bd58d52ee73a1599076abe221a31039cc94e2759a06a6732cbb0fc"}
Jul 14 16:19:11.565390 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:11.565285    2560 generic.go:356] "Generic (PLEG): container finished" podID="4f759301-008b-4906-8f05-70329aac89ab" containerID="7d2e1be782cbdc2c90eeae6a1c1922253b80a1966dfa118427dac1e52004e081" exitCode=0
Jul 14 16:19:11.565390 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:11.565330    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb" event={"ID":"4f759301-008b-4906-8f05-70329aac89ab","Type":"ContainerDied","Data":"7d2e1be782cbdc2c90eeae6a1c1922253b80a1966dfa118427dac1e52004e081"}
Jul 14 16:19:12.706970 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:12.706943    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb"
Jul 14 16:19:12.742059 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:12.742020    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f759301-008b-4906-8f05-70329aac89ab-maas-api-service-ca\") pod \"4f759301-008b-4906-8f05-70329aac89ab\" (UID: \"4f759301-008b-4906-8f05-70329aac89ab\") "
Jul 14 16:19:12.742354 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:12.742330    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f759301-008b-4906-8f05-70329aac89ab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f759301-008b-4906-8f05-70329aac89ab" (UID: "4f759301-008b-4906-8f05-70329aac89ab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:19:12.843348 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:12.843318    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f759301-008b-4906-8f05-70329aac89ab-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:19:13.576266 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:13.576235    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb"
Jul 14 16:19:13.576440 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:13.576237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb" event={"ID":"4f759301-008b-4906-8f05-70329aac89ab","Type":"ContainerDied","Data":"0ea22c9b14bd58d52ee73a1599076abe221a31039cc94e2759a06a6732cbb0fc"}
Jul 14 16:19:13.576440 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:13.576354    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ea22c9b14bd58d52ee73a1599076abe221a31039cc94e2759a06a6732cbb0fc"
Jul 14 16:19:20.511603 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.511566    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"]
Jul 14 16:19:20.512295 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.511834    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp" podUID="f92b97c0-408d-4571-97eb-b17d24187f3e" containerName="maas-api" containerID="cri-o://d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25" gracePeriod=30
Jul 14 16:19:20.766480 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.766398    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:19:20.814775 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.814749    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8b2n\" (UniqueName: \"kubernetes.io/projected/f92b97c0-408d-4571-97eb-b17d24187f3e-kube-api-access-n8b2n\") pod \"f92b97c0-408d-4571-97eb-b17d24187f3e\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") "
Jul 14 16:19:20.814939 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.814819    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f92b97c0-408d-4571-97eb-b17d24187f3e-maas-api-tls\") pod \"f92b97c0-408d-4571-97eb-b17d24187f3e\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") "
Jul 14 16:19:20.814939 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.814850    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92b97c0-408d-4571-97eb-b17d24187f3e-openshift-service-ca\") pod \"f92b97c0-408d-4571-97eb-b17d24187f3e\" (UID: \"f92b97c0-408d-4571-97eb-b17d24187f3e\") "
Jul 14 16:19:20.815261 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.815219    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f92b97c0-408d-4571-97eb-b17d24187f3e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f92b97c0-408d-4571-97eb-b17d24187f3e" (UID: "f92b97c0-408d-4571-97eb-b17d24187f3e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:19:20.816935 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.816909    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f92b97c0-408d-4571-97eb-b17d24187f3e-kube-api-access-n8b2n" (OuterVolumeSpecName: "kube-api-access-n8b2n") pod "f92b97c0-408d-4571-97eb-b17d24187f3e" (UID: "f92b97c0-408d-4571-97eb-b17d24187f3e"). InnerVolumeSpecName "kube-api-access-n8b2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:19:20.817024 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.816940    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f92b97c0-408d-4571-97eb-b17d24187f3e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f92b97c0-408d-4571-97eb-b17d24187f3e" (UID: "f92b97c0-408d-4571-97eb-b17d24187f3e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:19:20.916754 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.916723    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8b2n\" (UniqueName: \"kubernetes.io/projected/f92b97c0-408d-4571-97eb-b17d24187f3e-kube-api-access-n8b2n\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:19:20.916754 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.916749    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f92b97c0-408d-4571-97eb-b17d24187f3e-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:19:20.916754 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:20.916758    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f92b97c0-408d-4571-97eb-b17d24187f3e-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:19:21.626082 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:21.626050    2560 generic.go:356] "Generic (PLEG): container finished" podID="f92b97c0-408d-4571-97eb-b17d24187f3e" containerID="d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25" exitCode=0
Jul 14 16:19:21.626082 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:21.626091    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"
Jul 14 16:19:21.626588 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:21.626178    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp" event={"ID":"f92b97c0-408d-4571-97eb-b17d24187f3e","Type":"ContainerDied","Data":"d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25"}
Jul 14 16:19:21.626588 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:21.626226    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp" event={"ID":"f92b97c0-408d-4571-97eb-b17d24187f3e","Type":"ContainerDied","Data":"97ff90c685a4e33954e87bcd0c9164a5270ac38c1b8b7120d4c30747d905da3a"}
Jul 14 16:19:21.626588 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:21.626247    2560 scope.go:117] "RemoveContainer" containerID="d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25"
Jul 14 16:19:21.634986 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:21.634969    2560 scope.go:117] "RemoveContainer" containerID="d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25"
Jul 14 16:19:21.635222 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:19:21.635204    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25\": container with ID starting with d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25 not found: ID does not exist" containerID="d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25"
Jul 14 16:19:21.635289 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:21.635230    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25"} err="failed to get container status \"d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25\": rpc error: code = NotFound desc = could not find container \"d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25\": container with ID starting with d7fcebfd0b5e52c4be080ba7b498568f0766aba2caf758463779a88d4038ea25 not found: ID does not exist"
Jul 14 16:19:21.644380 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:21.644355    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"]
Jul 14 16:19:21.648471 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:21.648431    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b8cdb4a9-86bbc9f798-bb2fp"]
Jul 14 16:19:23.032632 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:19:23.032594    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f92b97c0-408d-4571-97eb-b17d24187f3e" path="/var/lib/kubelet/pods/f92b97c0-408d-4571-97eb-b17d24187f3e/volumes"
Jul 14 16:21:19.447959 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.447870    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"]
Jul 14 16:21:19.448518 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.448417    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f92b97c0-408d-4571-97eb-b17d24187f3e" containerName="maas-api"
Jul 14 16:21:19.448518 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.448432    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f92b97c0-408d-4571-97eb-b17d24187f3e" containerName="maas-api"
Jul 14 16:21:19.448518 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.448467    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerName="revoke-keys"
Jul 14 16:21:19.448518 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.448473    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="13a13abd-aa4f-4b22-aa31-d05edb365c9c" containerName="revoke-keys"
Jul 14 16:21:19.448518 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.448496    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f759301-008b-4906-8f05-70329aac89ab" containerName="revoke-keys"
Jul 14 16:21:19.448518 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.448501    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f759301-008b-4906-8f05-70329aac89ab" containerName="revoke-keys"
Jul 14 16:21:19.448716 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.448560    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f759301-008b-4906-8f05-70329aac89ab" containerName="revoke-keys"
Jul 14 16:21:19.448716 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.448570    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f92b97c0-408d-4571-97eb-b17d24187f3e" containerName="maas-api"
Jul 14 16:21:19.451651 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.451630    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:19.454463 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.454411    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 16:21:19.454587 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.454416    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d453a59f-serving-cert\""
Jul 14 16:21:19.455212 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.455189    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5kg5z\""
Jul 14 16:21:19.455350 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.455227    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 16:21:19.462420 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.462397    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"]
Jul 14 16:21:19.504204 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.504178    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfh8g\" (UniqueName: \"kubernetes.io/projected/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-kube-api-access-lfh8g\") pod \"maas-api-e2e-mt-d453a59f-585bfbd6d-ch226\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:19.504335 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.504230    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-maas-api-tls\") pod \"maas-api-e2e-mt-d453a59f-585bfbd6d-ch226\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:19.504335 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.504250    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-openshift-service-ca\") pod \"maas-api-e2e-mt-d453a59f-585bfbd6d-ch226\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:19.604943 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.604916    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfh8g\" (UniqueName: \"kubernetes.io/projected/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-kube-api-access-lfh8g\") pod \"maas-api-e2e-mt-d453a59f-585bfbd6d-ch226\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:19.605096 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.604961    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-maas-api-tls\") pod \"maas-api-e2e-mt-d453a59f-585bfbd6d-ch226\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:19.605096 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.604979    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-openshift-service-ca\") pod \"maas-api-e2e-mt-d453a59f-585bfbd6d-ch226\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:19.605096 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:21:19.605081    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-serving-cert: secret "maas-api-e2e-mt-d453a59f-serving-cert" not found
Jul 14 16:21:19.605270 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:21:19.605159    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-maas-api-tls podName:9263ba81-cbf1-4fe8-8f0a-5de304df7c98 nodeName:}" failed. No retries permitted until 2026-07-14 16:21:20.105136954 +0000 UTC m=+2063.624504366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-maas-api-tls") pod "maas-api-e2e-mt-d453a59f-585bfbd6d-ch226" (UID: "9263ba81-cbf1-4fe8-8f0a-5de304df7c98") : secret "maas-api-e2e-mt-d453a59f-serving-cert" not found
Jul 14 16:21:19.605588 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.605570    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-openshift-service-ca\") pod \"maas-api-e2e-mt-d453a59f-585bfbd6d-ch226\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:19.616217 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:19.616194    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfh8g\" (UniqueName: \"kubernetes.io/projected/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-kube-api-access-lfh8g\") pod \"maas-api-e2e-mt-d453a59f-585bfbd6d-ch226\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:20.109927 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:20.109896    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-maas-api-tls\") pod \"maas-api-e2e-mt-d453a59f-585bfbd6d-ch226\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:20.112466 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:20.112418    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-maas-api-tls\") pod \"maas-api-e2e-mt-d453a59f-585bfbd6d-ch226\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:20.363325 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:20.363235    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:20.503726 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:20.503698    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"]
Jul 14 16:21:20.505843 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:21:20.505814    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9263ba81_cbf1_4fe8_8f0a_5de304df7c98.slice/crio-85e5f35f554d011d05823ec20fc26ce3a62c414b54b4b8ce14ae85bdb4f72ac2 WatchSource:0}: Error finding container 85e5f35f554d011d05823ec20fc26ce3a62c414b54b4b8ce14ae85bdb4f72ac2: Status 404 returned error can't find the container with id 85e5f35f554d011d05823ec20fc26ce3a62c414b54b4b8ce14ae85bdb4f72ac2
Jul 14 16:21:20.507456 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:20.507425    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 16:21:21.258526 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:21.258491    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226" event={"ID":"9263ba81-cbf1-4fe8-8f0a-5de304df7c98","Type":"ContainerStarted","Data":"85e5f35f554d011d05823ec20fc26ce3a62c414b54b4b8ce14ae85bdb4f72ac2"}
Jul 14 16:21:22.264941 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:22.264904    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226" event={"ID":"9263ba81-cbf1-4fe8-8f0a-5de304df7c98","Type":"ContainerStarted","Data":"6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65"}
Jul 14 16:21:22.265329 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:22.264997    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:22.292723 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:22.292675    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226" podStartSLOduration=2.407629686 podStartE2EDuration="3.292661123s" totalImagesPullingTime="885.031437ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:21:22.292637471 +0000 UTC m=+2065.812004891" watchObservedRunningTime="2026-07-14 16:21:22.292661123 +0000 UTC m=+2065.812028566"
Jul 14 16:21:28.276215 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:28.276182    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:31.806131 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:31.806096    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr"]
Jul 14 16:21:31.810271 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:31.810250    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr"
Jul 14 16:21:31.813045 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:31.813016    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:21:31.815225 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:31.815199    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr"]
Jul 14 16:21:31.913094 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:31.913056    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a798393-5667-44d8-afba-9d430b12eef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d453a59f-6sktr\" (UID: \"9a798393-5667-44d8-afba-9d430b12eef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr"
Jul 14 16:21:32.014076 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:32.014048    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a798393-5667-44d8-afba-9d430b12eef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d453a59f-6sktr\" (UID: \"9a798393-5667-44d8-afba-9d430b12eef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr"
Jul 14 16:21:32.014721 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:32.014702    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a798393-5667-44d8-afba-9d430b12eef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d453a59f-6sktr\" (UID: \"9a798393-5667-44d8-afba-9d430b12eef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr"
Jul 14 16:21:32.123071 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:32.123043    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr"
Jul 14 16:21:32.463176 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:32.463147    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr"]
Jul 14 16:21:32.465165 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:21:32.465139    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a798393_5667_44d8_afba_9d430b12eef6.slice/crio-b8c89518fa466ba879a417e0d38053a2d69c0a250bfa27b27d24eddc4d013108 WatchSource:0}: Error finding container b8c89518fa466ba879a417e0d38053a2d69c0a250bfa27b27d24eddc4d013108: Status 404 returned error can't find the container with id b8c89518fa466ba879a417e0d38053a2d69c0a250bfa27b27d24eddc4d013108
Jul 14 16:21:33.328899 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:33.328800    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr" event={"ID":"9a798393-5667-44d8-afba-9d430b12eef6","Type":"ContainerStarted","Data":"dce28e48ef34f1e75cd0c63d984cdead7ec880324c6aa7656de994d8c8a648ed"}
Jul 14 16:21:33.328899 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:33.328893    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr" event={"ID":"9a798393-5667-44d8-afba-9d430b12eef6","Type":"ContainerStarted","Data":"b8c89518fa466ba879a417e0d38053a2d69c0a250bfa27b27d24eddc4d013108"}
Jul 14 16:21:33.355132 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:33.355082    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr" podStartSLOduration=2.355067985 podStartE2EDuration="2.355067985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:21:33.352888667 +0000 UTC m=+2076.872256087" watchObservedRunningTime="2026-07-14 16:21:33.355067985 +0000 UTC m=+2076.874435406"
Jul 14 16:21:34.335234 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:34.335201    2560 generic.go:356] "Generic (PLEG): container finished" podID="9a798393-5667-44d8-afba-9d430b12eef6" containerID="dce28e48ef34f1e75cd0c63d984cdead7ec880324c6aa7656de994d8c8a648ed" exitCode=0
Jul 14 16:21:34.335639 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:34.335287    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr" event={"ID":"9a798393-5667-44d8-afba-9d430b12eef6","Type":"ContainerDied","Data":"dce28e48ef34f1e75cd0c63d984cdead7ec880324c6aa7656de994d8c8a648ed"}
Jul 14 16:21:35.479846 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:35.479824    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr"
Jul 14 16:21:35.549028 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:35.549001    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a798393-5667-44d8-afba-9d430b12eef6-maas-api-service-ca\") pod \"9a798393-5667-44d8-afba-9d430b12eef6\" (UID: \"9a798393-5667-44d8-afba-9d430b12eef6\") "
Jul 14 16:21:35.549380 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:35.549359    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a798393-5667-44d8-afba-9d430b12eef6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9a798393-5667-44d8-afba-9d430b12eef6" (UID: "9a798393-5667-44d8-afba-9d430b12eef6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:21:35.650220 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:35.650153    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a798393-5667-44d8-afba-9d430b12eef6-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:21:36.347425 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:36.347398    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr"
Jul 14 16:21:36.347608 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:36.347436    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d453a59f-6sktr" event={"ID":"9a798393-5667-44d8-afba-9d430b12eef6","Type":"ContainerDied","Data":"b8c89518fa466ba879a417e0d38053a2d69c0a250bfa27b27d24eddc4d013108"}
Jul 14 16:21:36.347608 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:36.347501    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8c89518fa466ba879a417e0d38053a2d69c0a250bfa27b27d24eddc4d013108"
Jul 14 16:21:41.849563 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:41.849526    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"]
Jul 14 16:21:41.850037 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:41.849817    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226" podUID="9263ba81-cbf1-4fe8-8f0a-5de304df7c98" containerName="maas-api" containerID="cri-o://6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65" gracePeriod=30
Jul 14 16:21:42.097821 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.097798    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:42.212947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.212867    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-maas-api-tls\") pod \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") "
Jul 14 16:21:42.213092 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.212985    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-openshift-service-ca\") pod \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") "
Jul 14 16:21:42.213092 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.213027    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfh8g\" (UniqueName: \"kubernetes.io/projected/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-kube-api-access-lfh8g\") pod \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\" (UID: \"9263ba81-cbf1-4fe8-8f0a-5de304df7c98\") "
Jul 14 16:21:42.213353 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.213326    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9263ba81-cbf1-4fe8-8f0a-5de304df7c98" (UID: "9263ba81-cbf1-4fe8-8f0a-5de304df7c98"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:21:42.215244 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.215217    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9263ba81-cbf1-4fe8-8f0a-5de304df7c98" (UID: "9263ba81-cbf1-4fe8-8f0a-5de304df7c98"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:21:42.215244 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.215233    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-kube-api-access-lfh8g" (OuterVolumeSpecName: "kube-api-access-lfh8g") pod "9263ba81-cbf1-4fe8-8f0a-5de304df7c98" (UID: "9263ba81-cbf1-4fe8-8f0a-5de304df7c98"). InnerVolumeSpecName "kube-api-access-lfh8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:21:42.314371 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.314338    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:21:42.314371 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.314367    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfh8g\" (UniqueName: \"kubernetes.io/projected/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-kube-api-access-lfh8g\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:21:42.314371 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.314378    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9263ba81-cbf1-4fe8-8f0a-5de304df7c98-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:21:42.379142 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.379106    2560 generic.go:356] "Generic (PLEG): container finished" podID="9263ba81-cbf1-4fe8-8f0a-5de304df7c98" containerID="6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65" exitCode=0
Jul 14 16:21:42.379314 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.379148    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"
Jul 14 16:21:42.379314 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.379190    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226" event={"ID":"9263ba81-cbf1-4fe8-8f0a-5de304df7c98","Type":"ContainerDied","Data":"6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65"}
Jul 14 16:21:42.379314 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.379237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226" event={"ID":"9263ba81-cbf1-4fe8-8f0a-5de304df7c98","Type":"ContainerDied","Data":"85e5f35f554d011d05823ec20fc26ce3a62c414b54b4b8ce14ae85bdb4f72ac2"}
Jul 14 16:21:42.379314 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.379259    2560 scope.go:117] "RemoveContainer" containerID="6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65"
Jul 14 16:21:42.389103 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.389092    2560 scope.go:117] "RemoveContainer" containerID="6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65"
Jul 14 16:21:42.389345 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:21:42.389326    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65\": container with ID starting with 6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65 not found: ID does not exist" containerID="6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65"
Jul 14 16:21:42.389467 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.389349    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65"} err="failed to get container status \"6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65\": rpc error: code = NotFound desc = could not find container \"6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65\": container with ID starting with 6e3fb0b1e27e8d5d43976515ad5423a78bfcf515fc2a844797b5fb7bb820af65 not found: ID does not exist"
Jul 14 16:21:42.400827 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.400805    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"]
Jul 14 16:21:42.405157 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:42.405137    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d453a59f-585bfbd6d-ch226"]
Jul 14 16:21:43.032553 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:43.032524    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9263ba81-cbf1-4fe8-8f0a-5de304df7c98" path="/var/lib/kubelet/pods/9263ba81-cbf1-4fe8-8f0a-5de304df7c98/volumes"
Jul 14 16:21:57.074505 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:57.074479    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:21:57.081938 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:57.081917    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:21:57.138785 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:21:57.138757    2560 scope.go:117] "RemoveContainer" containerID="b3ac78693207b6beea30e36e9779e74a0ad2deb3fa520d3f98af0419cf7d65e3"
Jul 14 16:22:21.813829 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.813790    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"]
Jul 14 16:22:21.814471 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.814416    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9263ba81-cbf1-4fe8-8f0a-5de304df7c98" containerName="maas-api"
Jul 14 16:22:21.814471 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.814438    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9263ba81-cbf1-4fe8-8f0a-5de304df7c98" containerName="maas-api"
Jul 14 16:22:21.814611 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.814474    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a798393-5667-44d8-afba-9d430b12eef6" containerName="revoke-keys"
Jul 14 16:22:21.814611 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.814484    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a798393-5667-44d8-afba-9d430b12eef6" containerName="revoke-keys"
Jul 14 16:22:21.814611 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.814583    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9263ba81-cbf1-4fe8-8f0a-5de304df7c98" containerName="maas-api"
Jul 14 16:22:21.814611 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.814602    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a798393-5667-44d8-afba-9d430b12eef6" containerName="revoke-keys"
Jul 14 16:22:21.817875 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.817841    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:21.820259 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.820237    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ad3a3819-serving-cert\""
Jul 14 16:22:21.820684 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.820667    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5kg5z\""
Jul 14 16:22:21.820764 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.820723    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 16:22:21.820813 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.820769    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 16:22:21.828083 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.828058    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"]
Jul 14 16:22:21.983214 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.983174    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cc5f75a-fd55-4c9a-947e-d83319a46234-maas-api-tls\") pod \"maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:21.983214 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.983215    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc5f75a-fd55-4c9a-947e-d83319a46234-openshift-service-ca\") pod \"maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:21.983488 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:21.983325    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjcqb\" (UniqueName: \"kubernetes.io/projected/8cc5f75a-fd55-4c9a-947e-d83319a46234-kube-api-access-kjcqb\") pod \"maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:22.083854 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:22.083771    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cc5f75a-fd55-4c9a-947e-d83319a46234-maas-api-tls\") pod \"maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:22.083854 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:22.083815    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc5f75a-fd55-4c9a-947e-d83319a46234-openshift-service-ca\") pod \"maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:22.084091 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:22.083866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjcqb\" (UniqueName: \"kubernetes.io/projected/8cc5f75a-fd55-4c9a-947e-d83319a46234-kube-api-access-kjcqb\") pod \"maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:22.084091 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:22:22.083928    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-serving-cert: secret "maas-api-e2e-mt-ad3a3819-serving-cert" not found
Jul 14 16:22:22.084091 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:22:22.084003    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc5f75a-fd55-4c9a-947e-d83319a46234-maas-api-tls podName:8cc5f75a-fd55-4c9a-947e-d83319a46234 nodeName:}" failed. No retries permitted until 2026-07-14 16:22:22.583987524 +0000 UTC m=+2126.103354923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8cc5f75a-fd55-4c9a-947e-d83319a46234-maas-api-tls") pod "maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh" (UID: "8cc5f75a-fd55-4c9a-947e-d83319a46234") : secret "maas-api-e2e-mt-ad3a3819-serving-cert" not found
Jul 14 16:22:22.084574 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:22.084552    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc5f75a-fd55-4c9a-947e-d83319a46234-openshift-service-ca\") pod \"maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:22.101999 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:22.101968    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjcqb\" (UniqueName: \"kubernetes.io/projected/8cc5f75a-fd55-4c9a-947e-d83319a46234-kube-api-access-kjcqb\") pod \"maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:22.588853 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:22.588820    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cc5f75a-fd55-4c9a-947e-d83319a46234-maas-api-tls\") pod \"maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:22.591435 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:22.591413    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cc5f75a-fd55-4c9a-947e-d83319a46234-maas-api-tls\") pod \"maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:22.729892 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:22.729860    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:22.873400 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:22.873375    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"]
Jul 14 16:22:22.875230 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:22:22.875195    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc5f75a_fd55_4c9a_947e_d83319a46234.slice/crio-3c44b53970b48e077b09268289d38d1073e8a6a4cd2730b2efd77fde45ca963d WatchSource:0}: Error finding container 3c44b53970b48e077b09268289d38d1073e8a6a4cd2730b2efd77fde45ca963d: Status 404 returned error can't find the container with id 3c44b53970b48e077b09268289d38d1073e8a6a4cd2730b2efd77fde45ca963d
Jul 14 16:22:23.596324 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:23.596282    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh" event={"ID":"8cc5f75a-fd55-4c9a-947e-d83319a46234","Type":"ContainerStarted","Data":"52d284c15364a5f0ebaee6dc056bd98b0ece72b246e7631b1a3ef7c20f140c5a"}
Jul 14 16:22:23.596529 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:23.596342    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh" event={"ID":"8cc5f75a-fd55-4c9a-947e-d83319a46234","Type":"ContainerStarted","Data":"3c44b53970b48e077b09268289d38d1073e8a6a4cd2730b2efd77fde45ca963d"}
Jul 14 16:22:23.596529 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:23.596369    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:23.620483 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:23.620415    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh" podStartSLOduration=2.2348336189999998 podStartE2EDuration="2.620402324s" totalImagesPullingTime="385.568705ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:22:23.61880535 +0000 UTC m=+2127.138172771" watchObservedRunningTime="2026-07-14 16:22:23.620402324 +0000 UTC m=+2127.139769744"
Jul 14 16:22:29.607553 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:29.607511    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:22:43.298111 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.298078    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"]
Jul 14 16:22:43.301722 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.301702    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.305133 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.305111    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ad3a3819\"/\"e2e-lifecycle-model-ad3a3819-kserve-self-signed-certs\""
Jul 14 16:22:43.305252 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.305227    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ad3a3819\"/\"kube-root-ca.crt\""
Jul 14 16:22:43.305252 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.305230    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ad3a3819\"/\"default-dockercfg-k2bjq\""
Jul 14 16:22:43.305372 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.305228    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ad3a3819\"/\"openshift-service-ca.crt\""
Jul 14 16:22:43.312064 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.312040    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"]
Jul 14 16:22:43.381666 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.381636    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tmp-dir\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.381804 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.381672    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tls-certs\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.381804 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.381704    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-home\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.381804 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.381767    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-728jj\" (UniqueName: \"kubernetes.io/projected/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-kube-api-access-728jj\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.381913 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.381847    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-dshm\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.381913 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.381888    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-model-cache\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.482710 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.482673    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-728jj\" (UniqueName: \"kubernetes.io/projected/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-kube-api-access-728jj\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.482890 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.482737    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-dshm\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.482890 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.482767    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-model-cache\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.482890 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.482803    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tmp-dir\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.482890 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.482828    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tls-certs\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.482890 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.482866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-home\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.483232 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.483214    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-model-cache\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.483315 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.483296    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tmp-dir\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.483352 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.483319    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-home\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.485324 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.485294    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-dshm\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.485609 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.485589    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tls-certs\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.490678 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.490659    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-728jj\" (UniqueName: \"kubernetes.io/projected/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-kube-api-access-728jj\") pod \"e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") " pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.611185 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.611155    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:43.749581 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:43.749544    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"]
Jul 14 16:22:43.751668 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:22:43.751638    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod625c1c9c_3fcf_4e69_89d9_6f7d48ab9734.slice/crio-ed121f17839f97456c6bc689c3f5320f503d7a49e13a85ad51507b26c4c97a45 WatchSource:0}: Error finding container ed121f17839f97456c6bc689c3f5320f503d7a49e13a85ad51507b26c4c97a45: Status 404 returned error can't find the container with id ed121f17839f97456c6bc689c3f5320f503d7a49e13a85ad51507b26c4c97a45
Jul 14 16:22:44.710206 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:44.710169    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r" event={"ID":"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734","Type":"ContainerStarted","Data":"ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a"}
Jul 14 16:22:44.710616 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:44.710215    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r" event={"ID":"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734","Type":"ContainerStarted","Data":"ed121f17839f97456c6bc689c3f5320f503d7a49e13a85ad51507b26c4c97a45"}
Jul 14 16:22:44.728778 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:44.728729    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r" podStartSLOduration=1.728713723 podStartE2EDuration="1.728713723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:22:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:22:44.727327961 +0000 UTC m=+2148.246695384" watchObservedRunningTime="2026-07-14 16:22:44.728713723 +0000 UTC m=+2148.248081144"
Jul 14 16:22:53.612392 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:53.612308    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:53.612392 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:53.612353    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:53.624719 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:53.624691    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:53.771194 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:53.771165    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:22:57.161166 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:22:57.161137    2560 scope.go:117] "RemoveContainer" containerID="e869ca8db1528dc1a682d261abf3972f2f1916f3b4f2f0a31dbff612b0caea50"
Jul 14 16:23:40.981419 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:40.981381    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv"]
Jul 14 16:23:40.985019 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:40.985002    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv"
Jul 14 16:23:40.989774 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:40.989754    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:23:40.993816 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:40.993790    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv"]
Jul 14 16:23:41.120670 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:41.120634    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36b08e2-cdf0-4f48-9869-f5fd6e72b78b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv\" (UID: \"e36b08e2-cdf0-4f48-9869-f5fd6e72b78b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv"
Jul 14 16:23:41.221662 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:41.221629    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36b08e2-cdf0-4f48-9869-f5fd6e72b78b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv\" (UID: \"e36b08e2-cdf0-4f48-9869-f5fd6e72b78b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv"
Jul 14 16:23:41.222242 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:41.222222    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36b08e2-cdf0-4f48-9869-f5fd6e72b78b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv\" (UID: \"e36b08e2-cdf0-4f48-9869-f5fd6e72b78b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv"
Jul 14 16:23:41.296959 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:41.296878    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv"
Jul 14 16:23:41.437638 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:41.437606    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv"]
Jul 14 16:23:41.438833 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:23:41.438805    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode36b08e2_cdf0_4f48_9869_f5fd6e72b78b.slice/crio-5725ce20ac64e5521ff34db85347cd4f8c65efae8f9e482b37814755ad93c563 WatchSource:0}: Error finding container 5725ce20ac64e5521ff34db85347cd4f8c65efae8f9e482b37814755ad93c563: Status 404 returned error can't find the container with id 5725ce20ac64e5521ff34db85347cd4f8c65efae8f9e482b37814755ad93c563
Jul 14 16:23:42.013542 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:42.013458    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv" event={"ID":"e36b08e2-cdf0-4f48-9869-f5fd6e72b78b","Type":"ContainerStarted","Data":"ce4f9bce2c6b80427ed4950d66039748b1852b4a3d346ccfb144a86190bf7096"}
Jul 14 16:23:42.013542 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:42.013514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv" event={"ID":"e36b08e2-cdf0-4f48-9869-f5fd6e72b78b","Type":"ContainerStarted","Data":"5725ce20ac64e5521ff34db85347cd4f8c65efae8f9e482b37814755ad93c563"}
Jul 14 16:23:42.032492 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:42.032432    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv" podStartSLOduration=2.032418118 podStartE2EDuration="2.032418118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:23:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:23:42.029933651 +0000 UTC m=+2205.549301072" watchObservedRunningTime="2026-07-14 16:23:42.032418118 +0000 UTC m=+2205.551785540"
Jul 14 16:23:43.020951 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:43.020915    2560 generic.go:356] "Generic (PLEG): container finished" podID="e36b08e2-cdf0-4f48-9869-f5fd6e72b78b" containerID="ce4f9bce2c6b80427ed4950d66039748b1852b4a3d346ccfb144a86190bf7096" exitCode=0
Jul 14 16:23:43.021323 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:43.021004    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv" event={"ID":"e36b08e2-cdf0-4f48-9869-f5fd6e72b78b","Type":"ContainerDied","Data":"ce4f9bce2c6b80427ed4950d66039748b1852b4a3d346ccfb144a86190bf7096"}
Jul 14 16:23:44.162567 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:44.162543    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv"
Jul 14 16:23:44.251855 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:44.251829    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36b08e2-cdf0-4f48-9869-f5fd6e72b78b-maas-api-service-ca\") pod \"e36b08e2-cdf0-4f48-9869-f5fd6e72b78b\" (UID: \"e36b08e2-cdf0-4f48-9869-f5fd6e72b78b\") "
Jul 14 16:23:44.252180 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:44.252156    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e36b08e2-cdf0-4f48-9869-f5fd6e72b78b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e36b08e2-cdf0-4f48-9869-f5fd6e72b78b" (UID: "e36b08e2-cdf0-4f48-9869-f5fd6e72b78b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:23:44.353053 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:44.353028    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e36b08e2-cdf0-4f48-9869-f5fd6e72b78b-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:23:45.033670 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:45.033634    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv"
Jul 14 16:23:45.033670 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:45.033660    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv" event={"ID":"e36b08e2-cdf0-4f48-9869-f5fd6e72b78b","Type":"ContainerDied","Data":"5725ce20ac64e5521ff34db85347cd4f8c65efae8f9e482b37814755ad93c563"}
Jul 14 16:23:45.033946 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:45.033704    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5725ce20ac64e5521ff34db85347cd4f8c65efae8f9e482b37814755ad93c563"
Jul 14 16:23:51.828576 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:51.828538    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"]
Jul 14 16:23:51.829071 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:51.828748    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh" podUID="8cc5f75a-fd55-4c9a-947e-d83319a46234" containerName="maas-api" containerID="cri-o://52d284c15364a5f0ebaee6dc056bd98b0ece72b246e7631b1a3ef7c20f140c5a" gracePeriod=30
Jul 14 16:23:52.070206 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.070177    2560 generic.go:356] "Generic (PLEG): container finished" podID="8cc5f75a-fd55-4c9a-947e-d83319a46234" containerID="52d284c15364a5f0ebaee6dc056bd98b0ece72b246e7631b1a3ef7c20f140c5a" exitCode=0
Jul 14 16:23:52.070326 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.070225    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh" event={"ID":"8cc5f75a-fd55-4c9a-947e-d83319a46234","Type":"ContainerDied","Data":"52d284c15364a5f0ebaee6dc056bd98b0ece72b246e7631b1a3ef7c20f140c5a"}
Jul 14 16:23:52.070326 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.070275    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh" event={"ID":"8cc5f75a-fd55-4c9a-947e-d83319a46234","Type":"ContainerDied","Data":"3c44b53970b48e077b09268289d38d1073e8a6a4cd2730b2efd77fde45ca963d"}
Jul 14 16:23:52.070326 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.070291    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c44b53970b48e077b09268289d38d1073e8a6a4cd2730b2efd77fde45ca963d"
Jul 14 16:23:52.076324 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.076306    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:23:52.234713 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.234678    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjcqb\" (UniqueName: \"kubernetes.io/projected/8cc5f75a-fd55-4c9a-947e-d83319a46234-kube-api-access-kjcqb\") pod \"8cc5f75a-fd55-4c9a-947e-d83319a46234\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") "
Jul 14 16:23:52.234924 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.234802    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cc5f75a-fd55-4c9a-947e-d83319a46234-maas-api-tls\") pod \"8cc5f75a-fd55-4c9a-947e-d83319a46234\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") "
Jul 14 16:23:52.234924 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.234852    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc5f75a-fd55-4c9a-947e-d83319a46234-openshift-service-ca\") pod \"8cc5f75a-fd55-4c9a-947e-d83319a46234\" (UID: \"8cc5f75a-fd55-4c9a-947e-d83319a46234\") "
Jul 14 16:23:52.235203 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.235180    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cc5f75a-fd55-4c9a-947e-d83319a46234-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8cc5f75a-fd55-4c9a-947e-d83319a46234" (UID: "8cc5f75a-fd55-4c9a-947e-d83319a46234"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:23:52.236845 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.236811    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cc5f75a-fd55-4c9a-947e-d83319a46234-kube-api-access-kjcqb" (OuterVolumeSpecName: "kube-api-access-kjcqb") pod "8cc5f75a-fd55-4c9a-947e-d83319a46234" (UID: "8cc5f75a-fd55-4c9a-947e-d83319a46234"). InnerVolumeSpecName "kube-api-access-kjcqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:23:52.236953 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.236892    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cc5f75a-fd55-4c9a-947e-d83319a46234-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8cc5f75a-fd55-4c9a-947e-d83319a46234" (UID: "8cc5f75a-fd55-4c9a-947e-d83319a46234"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:23:52.336466 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.336416    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjcqb\" (UniqueName: \"kubernetes.io/projected/8cc5f75a-fd55-4c9a-947e-d83319a46234-kube-api-access-kjcqb\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:23:52.336466 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.336463    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cc5f75a-fd55-4c9a-947e-d83319a46234-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:23:52.336466 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:52.336473    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc5f75a-fd55-4c9a-947e-d83319a46234-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:23:53.075081 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:53.075056    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"
Jul 14 16:23:53.110628 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:53.110598    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"]
Jul 14 16:23:53.118454 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:53.118416    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad3a3819-94d78648f-sq2qh"]
Jul 14 16:23:55.032169 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:23:55.032133    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cc5f75a-fd55-4c9a-947e-d83319a46234" path="/var/lib/kubelet/pods/8cc5f75a-fd55-4c9a-947e-d83319a46234/volumes"
Jul 14 16:24:01.129231 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:01.129190    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"]
Jul 14 16:24:16.150256 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.150189    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r" podUID="625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" containerName="main" containerID="cri-o://ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a" gracePeriod=45
Jul 14 16:24:16.401907 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.401846    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:24:16.453323 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.453289    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-model-cache\") pod \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") "
Jul 14 16:24:16.453524 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.453389    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tls-certs\") pod \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") "
Jul 14 16:24:16.453524 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.453436    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tmp-dir\") pod \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") "
Jul 14 16:24:16.453644 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.453577    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-model-cache" (OuterVolumeSpecName: "model-cache") pod "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" (UID: "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:24:16.453644 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.453608    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-728jj\" (UniqueName: \"kubernetes.io/projected/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-kube-api-access-728jj\") pod \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") "
Jul 14 16:24:16.453752 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.453656    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-home\") pod \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") "
Jul 14 16:24:16.453752 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.453691    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-dshm\") pod \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\" (UID: \"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734\") "
Jul 14 16:24:16.453752 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.453707    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" (UID: "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:24:16.453986 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.453959    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-home" (OuterVolumeSpecName: "home") pod "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" (UID: "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:24:16.454137 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.454042    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-home\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:24:16.454137 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.454061    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-model-cache\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:24:16.454137 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.454075    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tmp-dir\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:24:16.455963 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.455941    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-dshm" (OuterVolumeSpecName: "dshm") pod "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" (UID: "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:24:16.456190 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.456170    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" (UID: "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:24:16.456289 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.456202    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-kube-api-access-728jj" (OuterVolumeSpecName: "kube-api-access-728jj") pod "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" (UID: "625c1c9c-3fcf-4e69-89d9-6f7d48ab9734"). InnerVolumeSpecName "kube-api-access-728jj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:24:16.555050 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.555015    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-tls-certs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:24:16.555050 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.555043    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-728jj\" (UniqueName: \"kubernetes.io/projected/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-kube-api-access-728jj\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:24:16.555050 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:16.555053    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734-dshm\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:24:17.200817 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:17.200733    2560 generic.go:356] "Generic (PLEG): container finished" podID="625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" containerID="ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a" exitCode=0
Jul 14 16:24:17.200817 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:17.200798    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"
Jul 14 16:24:17.201311 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:17.200823    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r" event={"ID":"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734","Type":"ContainerDied","Data":"ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a"}
Jul 14 16:24:17.201311 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:17.200868    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r" event={"ID":"625c1c9c-3fcf-4e69-89d9-6f7d48ab9734","Type":"ContainerDied","Data":"ed121f17839f97456c6bc689c3f5320f503d7a49e13a85ad51507b26c4c97a45"}
Jul 14 16:24:17.201311 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:17.200886    2560 scope.go:117] "RemoveContainer" containerID="ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a"
Jul 14 16:24:17.209611 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:17.209583    2560 scope.go:117] "RemoveContainer" containerID="ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a"
Jul 14 16:24:17.209870 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:24:17.209850    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a\": container with ID starting with ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a not found: ID does not exist" containerID="ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a"
Jul 14 16:24:17.209929 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:17.209879    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a"} err="failed to get container status \"ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a\": rpc error: code = NotFound desc = could not find container \"ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a\": container with ID starting with ebb8efdb04263928901d7a4c0357b2591589f5eedf2f1b0b4e34f55fffd5d04a not found: ID does not exist"
Jul 14 16:24:17.236003 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:17.235978    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"]
Jul 14 16:24:17.248533 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:17.248509    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ad3a3819/e2e-lifecycle-model-ad3a3819-kserve-6bf8cdb7f-v954r"]
Jul 14 16:24:19.032074 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:19.032041    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" path="/var/lib/kubelet/pods/625c1c9c-3fcf-4e69-89d9-6f7d48ab9734/volumes"
Jul 14 16:24:57.189245 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:24:57.189213    2560 scope.go:117] "RemoveContainer" containerID="5653bd4bdacd58613bd60f4fb8950e1b97daa25bdd5b522f11c60e064e9d2c23"
Jul 14 16:26:01.882581 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.882499    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"]
Jul 14 16:26:01.883138 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.883084    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cc5f75a-fd55-4c9a-947e-d83319a46234" containerName="maas-api"
Jul 14 16:26:01.883138 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.883104    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cc5f75a-fd55-4c9a-947e-d83319a46234" containerName="maas-api"
Jul 14 16:26:01.883138 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.883116    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" containerName="main"
Jul 14 16:26:01.883138 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.883126    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" containerName="main"
Jul 14 16:26:01.883371 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.883155    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36b08e2-cdf0-4f48-9869-f5fd6e72b78b" containerName="revoke-keys"
Jul 14 16:26:01.883371 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.883163    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36b08e2-cdf0-4f48-9869-f5fd6e72b78b" containerName="revoke-keys"
Jul 14 16:26:01.883371 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.883268    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e36b08e2-cdf0-4f48-9869-f5fd6e72b78b" containerName="revoke-keys"
Jul 14 16:26:01.883371 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.883280    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cc5f75a-fd55-4c9a-947e-d83319a46234" containerName="maas-api"
Jul 14 16:26:01.883371 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.883301    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="625c1c9c-3fcf-4e69-89d9-6f7d48ab9734" containerName="main"
Jul 14 16:26:01.886367 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.886347    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:01.888983 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.888964    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5kg5z\""
Jul 14 16:26:01.889371 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.889355    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a2d232-serving-cert\""
Jul 14 16:26:01.889938 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.889921    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 16:26:01.889938 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.889934    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 16:26:01.898857 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:01.898836    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"]
Jul 14 16:26:02.015076 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.015037    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb4787d-429c-433f-8da5-7fd56582853d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:02.015231 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.015183    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cb4787d-429c-433f-8da5-7fd56582853d-maas-api-tls\") pod \"maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:02.015286 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.015237    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knpff\" (UniqueName: \"kubernetes.io/projected/4cb4787d-429c-433f-8da5-7fd56582853d-kube-api-access-knpff\") pod \"maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:02.115970 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.115940    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knpff\" (UniqueName: \"kubernetes.io/projected/4cb4787d-429c-433f-8da5-7fd56582853d-kube-api-access-knpff\") pod \"maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:02.116128 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.115976    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb4787d-429c-433f-8da5-7fd56582853d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:02.116128 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.116053    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cb4787d-429c-433f-8da5-7fd56582853d-maas-api-tls\") pod \"maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:02.116213 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:26:02.116146    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-serving-cert: secret "maas-api-e2e-inf-a-a2d232-serving-cert" not found
Jul 14 16:26:02.116213 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:26:02.116194    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb4787d-429c-433f-8da5-7fd56582853d-maas-api-tls podName:4cb4787d-429c-433f-8da5-7fd56582853d nodeName:}" failed. No retries permitted until 2026-07-14 16:26:02.616178107 +0000 UTC m=+2346.135545506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4cb4787d-429c-433f-8da5-7fd56582853d-maas-api-tls") pod "maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf" (UID: "4cb4787d-429c-433f-8da5-7fd56582853d") : secret "maas-api-e2e-inf-a-a2d232-serving-cert" not found
Jul 14 16:26:02.116651 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.116631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb4787d-429c-433f-8da5-7fd56582853d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:02.125613 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.125591    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knpff\" (UniqueName: \"kubernetes.io/projected/4cb4787d-429c-433f-8da5-7fd56582853d-kube-api-access-knpff\") pod \"maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:02.621628 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.621596    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cb4787d-429c-433f-8da5-7fd56582853d-maas-api-tls\") pod \"maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:02.624198 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.624177    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cb4787d-429c-433f-8da5-7fd56582853d-maas-api-tls\") pod \"maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:02.797665 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.797628    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:02.939396 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:02.939374    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"]
Jul 14 16:26:02.941771 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:26:02.941739    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb4787d_429c_433f_8da5_7fd56582853d.slice/crio-12813e824ea57fb3b9519fe7f6f8b5cb2fdb2d789d382488d636893b84c801e8 WatchSource:0}: Error finding container 12813e824ea57fb3b9519fe7f6f8b5cb2fdb2d789d382488d636893b84c801e8: Status 404 returned error can't find the container with id 12813e824ea57fb3b9519fe7f6f8b5cb2fdb2d789d382488d636893b84c801e8
Jul 14 16:26:03.771395 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:03.771310    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf" event={"ID":"4cb4787d-429c-433f-8da5-7fd56582853d","Type":"ContainerStarted","Data":"d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f"}
Jul 14 16:26:03.771395 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:03.771383    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf" event={"ID":"4cb4787d-429c-433f-8da5-7fd56582853d","Type":"ContainerStarted","Data":"12813e824ea57fb3b9519fe7f6f8b5cb2fdb2d789d382488d636893b84c801e8"}
Jul 14 16:26:03.771610 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:03.771418    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:03.789832 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:03.789784    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf" podStartSLOduration=2.309519086 podStartE2EDuration="2.78976989s" totalImagesPullingTime="480.250804ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:26:03.789188633 +0000 UTC m=+2347.308556071" watchObservedRunningTime="2026-07-14 16:26:03.78976989 +0000 UTC m=+2347.309137311"
Jul 14 16:26:09.782797 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:09.782762    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:26:09.943488 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:09.943439    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"]
Jul 14 16:26:09.947344 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:09.947298    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:09.950251 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:09.950227    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a4a505-serving-cert\""
Jul 14 16:26:09.960260 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:09.960237    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"]
Jul 14 16:26:10.089862 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:10.089827    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2t5f\" (UniqueName: \"kubernetes.io/projected/9fc2c295-8017-46b4-bac6-dcb96dd6a272-kube-api-access-k2t5f\") pod \"maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:10.090025 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:10.089871    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fc2c295-8017-46b4-bac6-dcb96dd6a272-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:10.090025 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:10.089943    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fc2c295-8017-46b4-bac6-dcb96dd6a272-maas-api-tls\") pod \"maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:10.190747 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:10.190715    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2t5f\" (UniqueName: \"kubernetes.io/projected/9fc2c295-8017-46b4-bac6-dcb96dd6a272-kube-api-access-k2t5f\") pod \"maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:10.190747 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:10.190751    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fc2c295-8017-46b4-bac6-dcb96dd6a272-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:10.191017 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:10.190803    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fc2c295-8017-46b4-bac6-dcb96dd6a272-maas-api-tls\") pod \"maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:10.191017 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:26:10.190980    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-serving-cert: secret "maas-api-e2e-inf-b-a4a505-serving-cert" not found
Jul 14 16:26:10.191140 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:26:10.191065    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fc2c295-8017-46b4-bac6-dcb96dd6a272-maas-api-tls podName:9fc2c295-8017-46b4-bac6-dcb96dd6a272 nodeName:}" failed. No retries permitted until 2026-07-14 16:26:10.69104256 +0000 UTC m=+2354.210409978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9fc2c295-8017-46b4-bac6-dcb96dd6a272-maas-api-tls") pod "maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2" (UID: "9fc2c295-8017-46b4-bac6-dcb96dd6a272") : secret "maas-api-e2e-inf-b-a4a505-serving-cert" not found
Jul 14 16:26:10.191434 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:10.191415    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fc2c295-8017-46b4-bac6-dcb96dd6a272-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:10.199802 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:10.199778    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2t5f\" (UniqueName: \"kubernetes.io/projected/9fc2c295-8017-46b4-bac6-dcb96dd6a272-kube-api-access-k2t5f\") pod \"maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:10.696710 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:10.696679    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fc2c295-8017-46b4-bac6-dcb96dd6a272-maas-api-tls\") pod \"maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:10.699277 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:10.699244    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fc2c295-8017-46b4-bac6-dcb96dd6a272-maas-api-tls\") pod \"maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:10.858775 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:10.858524    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:11.014859 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:11.014826    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"]
Jul 14 16:26:11.015257 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:26:11.015231    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fc2c295_8017_46b4_bac6_dcb96dd6a272.slice/crio-c3b7b8e8e0598cc44673542cbd79a5afaf9059eb0c29a22c78680cf6084c6f41 WatchSource:0}: Error finding container c3b7b8e8e0598cc44673542cbd79a5afaf9059eb0c29a22c78680cf6084c6f41: Status 404 returned error can't find the container with id c3b7b8e8e0598cc44673542cbd79a5afaf9059eb0c29a22c78680cf6084c6f41
Jul 14 16:26:11.814256 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:11.814167    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2" event={"ID":"9fc2c295-8017-46b4-bac6-dcb96dd6a272","Type":"ContainerStarted","Data":"3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e"}
Jul 14 16:26:11.814256 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:11.814219    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2" event={"ID":"9fc2c295-8017-46b4-bac6-dcb96dd6a272","Type":"ContainerStarted","Data":"c3b7b8e8e0598cc44673542cbd79a5afaf9059eb0c29a22c78680cf6084c6f41"}
Jul 14 16:26:11.814523 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:11.814300    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:11.848833 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:11.848790    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2" podStartSLOduration=2.487098667 podStartE2EDuration="2.848775432s" totalImagesPullingTime="361.676765ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:26:11.848146724 +0000 UTC m=+2355.367514145" watchObservedRunningTime="2026-07-14 16:26:11.848775432 +0000 UTC m=+2355.368142883"
Jul 14 16:26:17.826741 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:17.826704    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:26:21.493765 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.493730    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"]
Jul 14 16:26:21.497536 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.497519    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.500112 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.500088    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a2d232\"/\"test-model-a2d232-kserve-self-signed-certs\""
Jul 14 16:26:21.501063 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.501043    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a2d232\"/\"default-dockercfg-pzst5\""
Jul 14 16:26:21.501184 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.501046    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a2d232\"/\"kube-root-ca.crt\""
Jul 14 16:26:21.501184 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.501093    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a2d232\"/\"openshift-service-ca.crt\""
Jul 14 16:26:21.506854 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.506828    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"]
Jul 14 16:26:21.593648 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.593616    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c442b\" (UniqueName: \"kubernetes.io/projected/83e73171-466e-4331-873e-4b220805f290-kube-api-access-c442b\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.593819 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.593660    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83e73171-466e-4331-873e-4b220805f290-tls-certs\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.593819 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.593695    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-home\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.593819 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.593763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-dshm\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.593819 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.593813    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-tmp-dir\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.593977 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.593856    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-model-cache\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.695161 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.695119    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-dshm\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.695333 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.695180    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-tmp-dir\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.695333 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.695227    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-model-cache\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.695333 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.695278    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c442b\" (UniqueName: \"kubernetes.io/projected/83e73171-466e-4331-873e-4b220805f290-kube-api-access-c442b\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.695333 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.695327    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83e73171-466e-4331-873e-4b220805f290-tls-certs\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.695600 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.695361    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-home\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.695697 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.695674    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-tmp-dir\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.695772 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.695713    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-model-cache\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.695772 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.695728    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-home\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.697574 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.697553    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-dshm\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.697863 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.697843    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83e73171-466e-4331-873e-4b220805f290-tls-certs\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.704167 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.704141    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c442b\" (UniqueName: \"kubernetes.io/projected/83e73171-466e-4331-873e-4b220805f290-kube-api-access-c442b\") pod \"test-model-a2d232-kserve-76fb4574bf-gk6fk\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") " pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.809581 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.809498    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:21.946947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:21.946921    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"]
Jul 14 16:26:21.948509 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:26:21.948480    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83e73171_466e_4331_873e_4b220805f290.slice/crio-69b7e422d2d52214959949f3d3eefaf53aa09e954138c307e02868f37c6803d8 WatchSource:0}: Error finding container 69b7e422d2d52214959949f3d3eefaf53aa09e954138c307e02868f37c6803d8: Status 404 returned error can't find the container with id 69b7e422d2d52214959949f3d3eefaf53aa09e954138c307e02868f37c6803d8
Jul 14 16:26:22.879012 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:22.878977    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk" event={"ID":"83e73171-466e-4331-873e-4b220805f290","Type":"ContainerStarted","Data":"11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc"}
Jul 14 16:26:22.879413 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:22.879019    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk" event={"ID":"83e73171-466e-4331-873e-4b220805f290","Type":"ContainerStarted","Data":"69b7e422d2d52214959949f3d3eefaf53aa09e954138c307e02868f37c6803d8"}
Jul 14 16:26:22.898084 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:22.898034    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk" podStartSLOduration=1.8980198499999998 podStartE2EDuration="1.89801985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:26:22.896251396 +0000 UTC m=+2366.415618818" watchObservedRunningTime="2026-07-14 16:26:22.89801985 +0000 UTC m=+2366.417387270"
Jul 14 16:26:31.810681 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:31.810648    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:31.811152 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:31.810696    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:31.823247 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:31.823216    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:31.938989 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:31.938959    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:26:57.108559 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:57.108530    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:26:57.123039 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:26:57.123017    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:27:11.795492 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.795437    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"]
Jul 14 16:27:11.799136 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.799117    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.802640 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.802582    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a4a505\"/\"kube-root-ca.crt\""
Jul 14 16:27:11.802728 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.802596    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a4a505\"/\"openshift-service-ca.crt\""
Jul 14 16:27:11.802806 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.802789    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a4a505\"/\"default-dockercfg-hgqxg\""
Jul 14 16:27:11.802853 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.802816    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a4a505\"/\"test-model-a4a505-kserve-self-signed-certs\""
Jul 14 16:27:11.809372 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.809349    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"]
Jul 14 16:27:11.875596 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.875566    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e75bb911-047c-49f9-804b-1d8d4cd14d82-tls-certs\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.875596 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.875598    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-home\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.875779 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.875617    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-model-cache\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.875779 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.875636    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-tmp-dir\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.875779 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.875724    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghvj8\" (UniqueName: \"kubernetes.io/projected/e75bb911-047c-49f9-804b-1d8d4cd14d82-kube-api-access-ghvj8\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.875779 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.875776    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-dshm\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.976279 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.976246    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-tmp-dir\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.976497 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.976303    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghvj8\" (UniqueName: \"kubernetes.io/projected/e75bb911-047c-49f9-804b-1d8d4cd14d82-kube-api-access-ghvj8\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.976497 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.976356    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-dshm\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.976497 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.976484    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e75bb911-047c-49f9-804b-1d8d4cd14d82-tls-certs\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.976638 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.976513    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-home\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.976638 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.976536    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-model-cache\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.976784 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.976748    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-tmp-dir\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.976909 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.976843    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-model-cache\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.976909 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.976879    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-home\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.978824 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.978804    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-dshm\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.979013 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.978994    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e75bb911-047c-49f9-804b-1d8d4cd14d82-tls-certs\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:11.985002 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:11.984981    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghvj8\" (UniqueName: \"kubernetes.io/projected/e75bb911-047c-49f9-804b-1d8d4cd14d82-kube-api-access-ghvj8\") pod \"test-model-a4a505-kserve-cbd5798fc-5sd58\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") " pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:12.108908 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:12.108872    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:12.246081 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:12.246046    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"]
Jul 14 16:27:12.248043 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:27:12.248011    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode75bb911_047c_49f9_804b_1d8d4cd14d82.slice/crio-6073cda58c0e6343eed946914c9c0d75d295d0c2b74425269e63b7bc67476ab3 WatchSource:0}: Error finding container 6073cda58c0e6343eed946914c9c0d75d295d0c2b74425269e63b7bc67476ab3: Status 404 returned error can't find the container with id 6073cda58c0e6343eed946914c9c0d75d295d0c2b74425269e63b7bc67476ab3
Jul 14 16:27:13.143189 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:13.143155    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58" event={"ID":"e75bb911-047c-49f9-804b-1d8d4cd14d82","Type":"ContainerStarted","Data":"cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa"}
Jul 14 16:27:13.143189 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:13.143195    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58" event={"ID":"e75bb911-047c-49f9-804b-1d8d4cd14d82","Type":"ContainerStarted","Data":"6073cda58c0e6343eed946914c9c0d75d295d0c2b74425269e63b7bc67476ab3"}
Jul 14 16:27:13.163261 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:13.163208    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58" podStartSLOduration=2.163193498 podStartE2EDuration="2.163193498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:27:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:27:13.16286264 +0000 UTC m=+2416.682230060" watchObservedRunningTime="2026-07-14 16:27:13.163193498 +0000 UTC m=+2416.682560949"
Jul 14 16:27:22.109307 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:22.109229    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:22.109307 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:22.109269    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:22.121947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:22.121922    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:22.202734 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:22.202713    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:33.827714 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:33.827684    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"]
Jul 14 16:27:34.829603 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:34.829572    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"]
Jul 14 16:27:45.657995 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:45.657953    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"]
Jul 14 16:27:45.658416 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:45.658198    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf" podUID="4cb4787d-429c-433f-8da5-7fd56582853d" containerName="maas-api" containerID="cri-o://d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f" gracePeriod=30
Jul 14 16:27:45.910741 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:45.910681    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:27:45.998591 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:45.998559    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cb4787d-429c-433f-8da5-7fd56582853d-maas-api-tls\") pod \"4cb4787d-429c-433f-8da5-7fd56582853d\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") "
Jul 14 16:27:45.998821 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:45.998652    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb4787d-429c-433f-8da5-7fd56582853d-openshift-service-ca\") pod \"4cb4787d-429c-433f-8da5-7fd56582853d\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") "
Jul 14 16:27:45.998821 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:45.998729    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knpff\" (UniqueName: \"kubernetes.io/projected/4cb4787d-429c-433f-8da5-7fd56582853d-kube-api-access-knpff\") pod \"4cb4787d-429c-433f-8da5-7fd56582853d\" (UID: \"4cb4787d-429c-433f-8da5-7fd56582853d\") "
Jul 14 16:27:45.999076 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:45.999044    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb4787d-429c-433f-8da5-7fd56582853d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4cb4787d-429c-433f-8da5-7fd56582853d" (UID: "4cb4787d-429c-433f-8da5-7fd56582853d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:27:46.000833 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.000791    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cb4787d-429c-433f-8da5-7fd56582853d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4cb4787d-429c-433f-8da5-7fd56582853d" (UID: "4cb4787d-429c-433f-8da5-7fd56582853d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:27:46.000939 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.000907    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb4787d-429c-433f-8da5-7fd56582853d-kube-api-access-knpff" (OuterVolumeSpecName: "kube-api-access-knpff") pod "4cb4787d-429c-433f-8da5-7fd56582853d" (UID: "4cb4787d-429c-433f-8da5-7fd56582853d"). InnerVolumeSpecName "kube-api-access-knpff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:27:46.099726 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.099693    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cb4787d-429c-433f-8da5-7fd56582853d-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:46.099726 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.099723    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb4787d-429c-433f-8da5-7fd56582853d-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:46.099726 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.099733    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knpff\" (UniqueName: \"kubernetes.io/projected/4cb4787d-429c-433f-8da5-7fd56582853d-kube-api-access-knpff\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:46.323331 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.323245    2560 generic.go:356] "Generic (PLEG): container finished" podID="4cb4787d-429c-433f-8da5-7fd56582853d" containerID="d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f" exitCode=0
Jul 14 16:27:46.323331 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.323293    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"
Jul 14 16:27:46.323331 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.323314    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf" event={"ID":"4cb4787d-429c-433f-8da5-7fd56582853d","Type":"ContainerDied","Data":"d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f"}
Jul 14 16:27:46.323605 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.323357    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf" event={"ID":"4cb4787d-429c-433f-8da5-7fd56582853d","Type":"ContainerDied","Data":"12813e824ea57fb3b9519fe7f6f8b5cb2fdb2d789d382488d636893b84c801e8"}
Jul 14 16:27:46.323605 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.323378    2560 scope.go:117] "RemoveContainer" containerID="d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f"
Jul 14 16:27:46.333231 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.333207    2560 scope.go:117] "RemoveContainer" containerID="d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f"
Jul 14 16:27:46.333497 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:27:46.333472    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f\": container with ID starting with d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f not found: ID does not exist" containerID="d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f"
Jul 14 16:27:46.333602 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.333504    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f"} err="failed to get container status \"d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f\": rpc error: code = NotFound desc = could not find container \"d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f\": container with ID starting with d2cbe9d2deb096330a30926fdbfafc096fbec12961c05b4230befeb08f94d72f not found: ID does not exist"
Jul 14 16:27:46.355051 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.355023    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"]
Jul 14 16:27:46.361428 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:46.361408    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a2d232-6989b57cff-nc9vf"]
Jul 14 16:27:47.032220 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:47.032187    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cb4787d-429c-433f-8da5-7fd56582853d" path="/var/lib/kubelet/pods/4cb4787d-429c-433f-8da5-7fd56582853d/volumes"
Jul 14 16:27:48.848543 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:48.848497    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk" podUID="83e73171-466e-4331-873e-4b220805f290" containerName="main" containerID="cri-o://11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc" gracePeriod=45
Jul 14 16:27:49.107729 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.107660    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:27:49.127520 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.127496    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-home\") pod \"83e73171-466e-4331-873e-4b220805f290\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") "
Jul 14 16:27:49.130342 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.127551    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-model-cache\") pod \"83e73171-466e-4331-873e-4b220805f290\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") "
Jul 14 16:27:49.130342 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.127630    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c442b\" (UniqueName: \"kubernetes.io/projected/83e73171-466e-4331-873e-4b220805f290-kube-api-access-c442b\") pod \"83e73171-466e-4331-873e-4b220805f290\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") "
Jul 14 16:27:49.130342 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.127702    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-tmp-dir\") pod \"83e73171-466e-4331-873e-4b220805f290\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") "
Jul 14 16:27:49.130342 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.127726    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-dshm\") pod \"83e73171-466e-4331-873e-4b220805f290\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") "
Jul 14 16:27:49.130342 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.127761    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-home" (OuterVolumeSpecName: "home") pod "83e73171-466e-4331-873e-4b220805f290" (UID: "83e73171-466e-4331-873e-4b220805f290"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:27:49.130342 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.127973    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "83e73171-466e-4331-873e-4b220805f290" (UID: "83e73171-466e-4331-873e-4b220805f290"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:27:49.130342 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.128021    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83e73171-466e-4331-873e-4b220805f290-tls-certs\") pod \"83e73171-466e-4331-873e-4b220805f290\" (UID: \"83e73171-466e-4331-873e-4b220805f290\") "
Jul 14 16:27:49.130342 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.128337    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-tmp-dir\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:49.130342 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.128351    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-home\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:49.130342 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.128906    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-model-cache" (OuterVolumeSpecName: "model-cache") pod "83e73171-466e-4331-873e-4b220805f290" (UID: "83e73171-466e-4331-873e-4b220805f290"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:27:49.131382 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.131346    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-dshm" (OuterVolumeSpecName: "dshm") pod "83e73171-466e-4331-873e-4b220805f290" (UID: "83e73171-466e-4331-873e-4b220805f290"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:27:49.132071 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.132047    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83e73171-466e-4331-873e-4b220805f290-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "83e73171-466e-4331-873e-4b220805f290" (UID: "83e73171-466e-4331-873e-4b220805f290"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:27:49.132545 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.132525    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83e73171-466e-4331-873e-4b220805f290-kube-api-access-c442b" (OuterVolumeSpecName: "kube-api-access-c442b") pod "83e73171-466e-4331-873e-4b220805f290" (UID: "83e73171-466e-4331-873e-4b220805f290"). InnerVolumeSpecName "kube-api-access-c442b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:27:49.229227 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.229198    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-model-cache\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:49.229227 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.229223    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c442b\" (UniqueName: \"kubernetes.io/projected/83e73171-466e-4331-873e-4b220805f290-kube-api-access-c442b\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:49.229227 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.229234    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83e73171-466e-4331-873e-4b220805f290-dshm\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:49.229486 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.229243    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83e73171-466e-4331-873e-4b220805f290-tls-certs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:49.341604 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.341573    2560 generic.go:356] "Generic (PLEG): container finished" podID="83e73171-466e-4331-873e-4b220805f290" containerID="11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc" exitCode=0
Jul 14 16:27:49.341766 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.341616    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"
Jul 14 16:27:49.341766 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.341620    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk" event={"ID":"83e73171-466e-4331-873e-4b220805f290","Type":"ContainerDied","Data":"11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc"}
Jul 14 16:27:49.341766 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.341653    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk" event={"ID":"83e73171-466e-4331-873e-4b220805f290","Type":"ContainerDied","Data":"69b7e422d2d52214959949f3d3eefaf53aa09e954138c307e02868f37c6803d8"}
Jul 14 16:27:49.341766 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.341672    2560 scope.go:117] "RemoveContainer" containerID="11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc"
Jul 14 16:27:49.351096 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.351078    2560 scope.go:117] "RemoveContainer" containerID="11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc"
Jul 14 16:27:49.351349 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:27:49.351330    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc\": container with ID starting with 11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc not found: ID does not exist" containerID="11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc"
Jul 14 16:27:49.351402 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.351358    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc"} err="failed to get container status \"11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc\": rpc error: code = NotFound desc = could not find container \"11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc\": container with ID starting with 11678b2dc25b84d8ea421d9ccf852a605d6b046f46850f06f153d8549f6216dc not found: ID does not exist"
Jul 14 16:27:49.367094 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.367039    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"]
Jul 14 16:27:49.369042 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.369024    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a2d232/test-model-a2d232-kserve-76fb4574bf-gk6fk"]
Jul 14 16:27:49.850371 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:49.850324    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58" podUID="e75bb911-047c-49f9-804b-1d8d4cd14d82" containerName="main" containerID="cri-o://cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa" gracePeriod=45
Jul 14 16:27:50.109846 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.109781    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:50.138384 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.138360    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-tmp-dir\") pod \"e75bb911-047c-49f9-804b-1d8d4cd14d82\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") "
Jul 14 16:27:50.138526 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.138401    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-home\") pod \"e75bb911-047c-49f9-804b-1d8d4cd14d82\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") "
Jul 14 16:27:50.138526 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.138441    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-model-cache\") pod \"e75bb911-047c-49f9-804b-1d8d4cd14d82\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") "
Jul 14 16:27:50.138526 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.138500    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e75bb911-047c-49f9-804b-1d8d4cd14d82-tls-certs\") pod \"e75bb911-047c-49f9-804b-1d8d4cd14d82\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") "
Jul 14 16:27:50.138697 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.138535    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-dshm\") pod \"e75bb911-047c-49f9-804b-1d8d4cd14d82\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") "
Jul 14 16:27:50.138697 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.138607    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghvj8\" (UniqueName: \"kubernetes.io/projected/e75bb911-047c-49f9-804b-1d8d4cd14d82-kube-api-access-ghvj8\") pod \"e75bb911-047c-49f9-804b-1d8d4cd14d82\" (UID: \"e75bb911-047c-49f9-804b-1d8d4cd14d82\") "
Jul 14 16:27:50.138697 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.138672    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e75bb911-047c-49f9-804b-1d8d4cd14d82" (UID: "e75bb911-047c-49f9-804b-1d8d4cd14d82"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:27:50.138848 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.138684    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-home" (OuterVolumeSpecName: "home") pod "e75bb911-047c-49f9-804b-1d8d4cd14d82" (UID: "e75bb911-047c-49f9-804b-1d8d4cd14d82"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:27:50.138848 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.138771    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-model-cache" (OuterVolumeSpecName: "model-cache") pod "e75bb911-047c-49f9-804b-1d8d4cd14d82" (UID: "e75bb911-047c-49f9-804b-1d8d4cd14d82"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:27:50.139001 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.138983    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-tmp-dir\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:50.139056 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.139006    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-home\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:50.139056 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.139018    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-model-cache\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:50.140848 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.140824    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-dshm" (OuterVolumeSpecName: "dshm") pod "e75bb911-047c-49f9-804b-1d8d4cd14d82" (UID: "e75bb911-047c-49f9-804b-1d8d4cd14d82"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:27:50.141275 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.141256    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e75bb911-047c-49f9-804b-1d8d4cd14d82-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e75bb911-047c-49f9-804b-1d8d4cd14d82" (UID: "e75bb911-047c-49f9-804b-1d8d4cd14d82"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:27:50.141335 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.141268    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e75bb911-047c-49f9-804b-1d8d4cd14d82-kube-api-access-ghvj8" (OuterVolumeSpecName: "kube-api-access-ghvj8") pod "e75bb911-047c-49f9-804b-1d8d4cd14d82" (UID: "e75bb911-047c-49f9-804b-1d8d4cd14d82"). InnerVolumeSpecName "kube-api-access-ghvj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:27:50.239888 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.239862    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e75bb911-047c-49f9-804b-1d8d4cd14d82-tls-certs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:50.239888 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.239890    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e75bb911-047c-49f9-804b-1d8d4cd14d82-dshm\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:50.240047 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.239904    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghvj8\" (UniqueName: \"kubernetes.io/projected/e75bb911-047c-49f9-804b-1d8d4cd14d82-kube-api-access-ghvj8\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:27:50.349595 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.349558    2560 generic.go:356] "Generic (PLEG): container finished" podID="e75bb911-047c-49f9-804b-1d8d4cd14d82" containerID="cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa" exitCode=0
Jul 14 16:27:50.349747 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.349605    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"
Jul 14 16:27:50.349747 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.349615    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58" event={"ID":"e75bb911-047c-49f9-804b-1d8d4cd14d82","Type":"ContainerDied","Data":"cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa"}
Jul 14 16:27:50.349747 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.349652    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58" event={"ID":"e75bb911-047c-49f9-804b-1d8d4cd14d82","Type":"ContainerDied","Data":"6073cda58c0e6343eed946914c9c0d75d295d0c2b74425269e63b7bc67476ab3"}
Jul 14 16:27:50.349747 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.349674    2560 scope.go:117] "RemoveContainer" containerID="cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa"
Jul 14 16:27:50.364968 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.364226    2560 scope.go:117] "RemoveContainer" containerID="cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa"
Jul 14 16:27:50.364968 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:27:50.364658    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa\": container with ID starting with cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa not found: ID does not exist" containerID="cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa"
Jul 14 16:27:50.364968 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.364699    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa"} err="failed to get container status \"cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa\": rpc error: code = NotFound desc = could not find container \"cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa\": container with ID starting with cae508639636783bc170f692b6e97ffee3be20815a3f4c6a59aa033794c772aa not found: ID does not exist"
Jul 14 16:27:50.383790 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.383766    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"]
Jul 14 16:27:50.401158 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:50.401131    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a4a505/test-model-a4a505-kserve-cbd5798fc-5sd58"]
Jul 14 16:27:51.032308 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:51.032262    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83e73171-466e-4331-873e-4b220805f290" path="/var/lib/kubelet/pods/83e73171-466e-4331-873e-4b220805f290/volumes"
Jul 14 16:27:51.032892 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:27:51.032870    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e75bb911-047c-49f9-804b-1d8d4cd14d82" path="/var/lib/kubelet/pods/e75bb911-047c-49f9-804b-1d8d4cd14d82/volumes"
Jul 14 16:28:01.245423 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.245385    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp"]
Jul 14 16:28:01.245830 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.245815    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e75bb911-047c-49f9-804b-1d8d4cd14d82" containerName="main"
Jul 14 16:28:01.245830 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.245831    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e75bb911-047c-49f9-804b-1d8d4cd14d82" containerName="main"
Jul 14 16:28:01.245909 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.245843    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83e73171-466e-4331-873e-4b220805f290" containerName="main"
Jul 14 16:28:01.245909 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.245848    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="83e73171-466e-4331-873e-4b220805f290" containerName="main"
Jul 14 16:28:01.245909 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.245863    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cb4787d-429c-433f-8da5-7fd56582853d" containerName="maas-api"
Jul 14 16:28:01.245909 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.245870    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cb4787d-429c-433f-8da5-7fd56582853d" containerName="maas-api"
Jul 14 16:28:01.246028 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.245927    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cb4787d-429c-433f-8da5-7fd56582853d" containerName="maas-api"
Jul 14 16:28:01.246028 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.245935    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e75bb911-047c-49f9-804b-1d8d4cd14d82" containerName="main"
Jul 14 16:28:01.246028 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.245943    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="83e73171-466e-4331-873e-4b220805f290" containerName="main"
Jul 14 16:28:01.248778 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.248761    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp"
Jul 14 16:28:01.251437 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.251416    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:28:01.257911 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.257890    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp"]
Jul 14 16:28:01.341093 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.341056    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82c61249-f6a9-4550-9c9e-b88e7e046b73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a4a505-782sp\" (UID: \"82c61249-f6a9-4550-9c9e-b88e7e046b73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp"
Jul 14 16:28:01.441623 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.441594    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82c61249-f6a9-4550-9c9e-b88e7e046b73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a4a505-782sp\" (UID: \"82c61249-f6a9-4550-9c9e-b88e7e046b73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp"
Jul 14 16:28:01.442191 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.442168    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82c61249-f6a9-4550-9c9e-b88e7e046b73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a4a505-782sp\" (UID: \"82c61249-f6a9-4550-9c9e-b88e7e046b73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp"
Jul 14 16:28:01.560912 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.560826    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp"
Jul 14 16:28:01.700544 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:01.700511    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp"]
Jul 14 16:28:01.702071 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:28:01.702037    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c61249_f6a9_4550_9c9e_b88e7e046b73.slice/crio-c0e8c11f6e3633ba6040927b4f67510c52695c9ef63d7c46df72c0cd6cfe43c8 WatchSource:0}: Error finding container c0e8c11f6e3633ba6040927b4f67510c52695c9ef63d7c46df72c0cd6cfe43c8: Status 404 returned error can't find the container with id c0e8c11f6e3633ba6040927b4f67510c52695c9ef63d7c46df72c0cd6cfe43c8
Jul 14 16:28:02.415697 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:02.415667    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp" event={"ID":"82c61249-f6a9-4550-9c9e-b88e7e046b73","Type":"ContainerStarted","Data":"7df49fcb9f2589eddce65ddf56047239a3fcec09d8167b15c3ca77a4934773c1"}
Jul 14 16:28:02.415996 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:02.415752    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp" event={"ID":"82c61249-f6a9-4550-9c9e-b88e7e046b73","Type":"ContainerStarted","Data":"c0e8c11f6e3633ba6040927b4f67510c52695c9ef63d7c46df72c0cd6cfe43c8"}
Jul 14 16:28:03.422375 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:03.422340    2560 generic.go:356] "Generic (PLEG): container finished" podID="82c61249-f6a9-4550-9c9e-b88e7e046b73" containerID="7df49fcb9f2589eddce65ddf56047239a3fcec09d8167b15c3ca77a4934773c1" exitCode=0
Jul 14 16:28:03.422796 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:03.422426    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp" event={"ID":"82c61249-f6a9-4550-9c9e-b88e7e046b73","Type":"ContainerDied","Data":"7df49fcb9f2589eddce65ddf56047239a3fcec09d8167b15c3ca77a4934773c1"}
Jul 14 16:28:03.555997 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:03.555974    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp"
Jul 14 16:28:03.662572 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:03.662541    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82c61249-f6a9-4550-9c9e-b88e7e046b73-maas-api-service-ca\") pod \"82c61249-f6a9-4550-9c9e-b88e7e046b73\" (UID: \"82c61249-f6a9-4550-9c9e-b88e7e046b73\") "
Jul 14 16:28:03.662910 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:03.662887    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c61249-f6a9-4550-9c9e-b88e7e046b73-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "82c61249-f6a9-4550-9c9e-b88e7e046b73" (UID: "82c61249-f6a9-4550-9c9e-b88e7e046b73"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:28:03.763915 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:03.763828    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82c61249-f6a9-4550-9c9e-b88e7e046b73-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:28:04.429420 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:04.429390    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp"
Jul 14 16:28:04.429831 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:04.429432    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a4a505-782sp" event={"ID":"82c61249-f6a9-4550-9c9e-b88e7e046b73","Type":"ContainerDied","Data":"c0e8c11f6e3633ba6040927b4f67510c52695c9ef63d7c46df72c0cd6cfe43c8"}
Jul 14 16:28:04.429831 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:04.429499    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0e8c11f6e3633ba6040927b4f67510c52695c9ef63d7c46df72c0cd6cfe43c8"
Jul 14 16:28:11.704792 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:11.704756    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"]
Jul 14 16:28:11.705323 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:11.704984    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2" podUID="9fc2c295-8017-46b4-bac6-dcb96dd6a272" containerName="maas-api" containerID="cri-o://3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e" gracePeriod=30
Jul 14 16:28:11.959168 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:11.959111    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:28:12.045314 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.045286    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fc2c295-8017-46b4-bac6-dcb96dd6a272-maas-api-tls\") pod \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") "
Jul 14 16:28:12.045544 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.045411    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2t5f\" (UniqueName: \"kubernetes.io/projected/9fc2c295-8017-46b4-bac6-dcb96dd6a272-kube-api-access-k2t5f\") pod \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") "
Jul 14 16:28:12.045544 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.045440    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fc2c295-8017-46b4-bac6-dcb96dd6a272-openshift-service-ca\") pod \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\" (UID: \"9fc2c295-8017-46b4-bac6-dcb96dd6a272\") "
Jul 14 16:28:12.045870 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.045844    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fc2c295-8017-46b4-bac6-dcb96dd6a272-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9fc2c295-8017-46b4-bac6-dcb96dd6a272" (UID: "9fc2c295-8017-46b4-bac6-dcb96dd6a272"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:28:12.047636 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.047613    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fc2c295-8017-46b4-bac6-dcb96dd6a272-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9fc2c295-8017-46b4-bac6-dcb96dd6a272" (UID: "9fc2c295-8017-46b4-bac6-dcb96dd6a272"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:28:12.047732 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.047689    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fc2c295-8017-46b4-bac6-dcb96dd6a272-kube-api-access-k2t5f" (OuterVolumeSpecName: "kube-api-access-k2t5f") pod "9fc2c295-8017-46b4-bac6-dcb96dd6a272" (UID: "9fc2c295-8017-46b4-bac6-dcb96dd6a272"). InnerVolumeSpecName "kube-api-access-k2t5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:28:12.146524 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.146492    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2t5f\" (UniqueName: \"kubernetes.io/projected/9fc2c295-8017-46b4-bac6-dcb96dd6a272-kube-api-access-k2t5f\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:28:12.146524 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.146518    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fc2c295-8017-46b4-bac6-dcb96dd6a272-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:28:12.146524 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.146527    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fc2c295-8017-46b4-bac6-dcb96dd6a272-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:28:12.472758 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.472723    2560 generic.go:356] "Generic (PLEG): container finished" podID="9fc2c295-8017-46b4-bac6-dcb96dd6a272" containerID="3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e" exitCode=0
Jul 14 16:28:12.472758 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.472760    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2" event={"ID":"9fc2c295-8017-46b4-bac6-dcb96dd6a272","Type":"ContainerDied","Data":"3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e"}
Jul 14 16:28:12.472758 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.472765    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"
Jul 14 16:28:12.473035 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.472798    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2" event={"ID":"9fc2c295-8017-46b4-bac6-dcb96dd6a272","Type":"ContainerDied","Data":"c3b7b8e8e0598cc44673542cbd79a5afaf9059eb0c29a22c78680cf6084c6f41"}
Jul 14 16:28:12.473035 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.472822    2560 scope.go:117] "RemoveContainer" containerID="3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e"
Jul 14 16:28:12.482265 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.482247    2560 scope.go:117] "RemoveContainer" containerID="3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e"
Jul 14 16:28:12.482542 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:28:12.482523    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e\": container with ID starting with 3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e not found: ID does not exist" containerID="3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e"
Jul 14 16:28:12.482614 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.482554    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e"} err="failed to get container status \"3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e\": rpc error: code = NotFound desc = could not find container \"3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e\": container with ID starting with 3fe2de3dd74c95124db6f9e84c5504c5bd257635a4561f6e3a0ec21855e85b7e not found: ID does not exist"
Jul 14 16:28:12.495828 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.495804    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"]
Jul 14 16:28:12.499755 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:12.499734    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a4a505-c88d5d55b-djtm2"]
Jul 14 16:28:13.032043 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:13.032001    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fc2c295-8017-46b4-bac6-dcb96dd6a272" path="/var/lib/kubelet/pods/9fc2c295-8017-46b4-bac6-dcb96dd6a272/volumes"
Jul 14 16:28:30.848977 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.848936    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"]
Jul 14 16:28:30.849439 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.849336    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fc2c295-8017-46b4-bac6-dcb96dd6a272" containerName="maas-api"
Jul 14 16:28:30.849439 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.849346    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fc2c295-8017-46b4-bac6-dcb96dd6a272" containerName="maas-api"
Jul 14 16:28:30.849439 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.849356    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82c61249-f6a9-4550-9c9e-b88e7e046b73" containerName="revoke-keys"
Jul 14 16:28:30.849439 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.849362    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="82c61249-f6a9-4550-9c9e-b88e7e046b73" containerName="revoke-keys"
Jul 14 16:28:30.849439 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.849433    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fc2c295-8017-46b4-bac6-dcb96dd6a272" containerName="maas-api"
Jul 14 16:28:30.849729 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.849470    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="82c61249-f6a9-4550-9c9e-b88e7e046b73" containerName="revoke-keys"
Jul 14 16:28:30.853944 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.853924    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:30.857097 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.857075    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 16:28:30.858135 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.858110    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 16:28:30.858240 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.858157    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5kg5z\""
Jul 14 16:28:30.858240 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.858172    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d7e901-serving-cert\""
Jul 14 16:28:30.864924 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.864905    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"]
Jul 14 16:28:30.913759 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.913724    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xc4f\" (UniqueName: \"kubernetes.io/projected/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-kube-api-access-4xc4f\") pod \"maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:30.913921 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.913777    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-maas-api-tls\") pod \"maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:30.913921 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:30.913831    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-openshift-service-ca\") pod \"maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:31.014895 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:31.014863    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-openshift-service-ca\") pod \"maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:31.015098 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:31.014960    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xc4f\" (UniqueName: \"kubernetes.io/projected/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-kube-api-access-4xc4f\") pod \"maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:31.015098 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:31.014994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-maas-api-tls\") pod \"maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:31.015098 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:28:31.015093    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-serving-cert: secret "maas-api-e2e-api-a-d7e901-serving-cert" not found
Jul 14 16:28:31.015263 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:28:31.015158    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-maas-api-tls podName:a7ddc10a-fecc-4820-bcff-0b3641de6b6f nodeName:}" failed. No retries permitted until 2026-07-14 16:28:31.515140757 +0000 UTC m=+2495.034508182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-maas-api-tls") pod "maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf" (UID: "a7ddc10a-fecc-4820-bcff-0b3641de6b6f") : secret "maas-api-e2e-api-a-d7e901-serving-cert" not found
Jul 14 16:28:31.015528 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:31.015507    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-openshift-service-ca\") pod \"maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:31.028787 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:31.028764    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xc4f\" (UniqueName: \"kubernetes.io/projected/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-kube-api-access-4xc4f\") pod \"maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:31.520398 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:31.520356    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-maas-api-tls\") pod \"maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:31.522971 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:31.522950    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-maas-api-tls\") pod \"maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:31.766972 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:31.766940    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:31.912168 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:31.912143    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"]
Jul 14 16:28:31.914235 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:28:31.914202    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ddc10a_fecc_4820_bcff_0b3641de6b6f.slice/crio-cccd7fd061b24b59bfbaf74994651cad75730369d2b62e29c4a4bd7448995975 WatchSource:0}: Error finding container cccd7fd061b24b59bfbaf74994651cad75730369d2b62e29c4a4bd7448995975: Status 404 returned error can't find the container with id cccd7fd061b24b59bfbaf74994651cad75730369d2b62e29c4a4bd7448995975
Jul 14 16:28:31.915660 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:31.915640    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 16:28:32.581114 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:32.581064    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf" event={"ID":"a7ddc10a-fecc-4820-bcff-0b3641de6b6f","Type":"ContainerStarted","Data":"75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c"}
Jul 14 16:28:32.581282 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:32.581134    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf" event={"ID":"a7ddc10a-fecc-4820-bcff-0b3641de6b6f","Type":"ContainerStarted","Data":"cccd7fd061b24b59bfbaf74994651cad75730369d2b62e29c4a4bd7448995975"}
Jul 14 16:28:32.581282 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:32.581166    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:32.598913 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:32.598859    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf" podStartSLOduration=2.106850006 podStartE2EDuration="2.598845971s" totalImagesPullingTime="491.995965ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:28:32.597979237 +0000 UTC m=+2496.117346671" watchObservedRunningTime="2026-07-14 16:28:32.598845971 +0000 UTC m=+2496.118213392"
Jul 14 16:28:38.590942 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.590916    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:28:38.852605 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.852515    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"]
Jul 14 16:28:38.856287 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.856262    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:38.858774 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.858749    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-c40588-serving-cert\""
Jul 14 16:28:38.865212 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.865192    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"]
Jul 14 16:28:38.896775 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.896745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f74vs\" (UniqueName: \"kubernetes.io/projected/fdd2da80-9a6e-45ee-a107-02516c0540b6-kube-api-access-f74vs\") pod \"maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:38.896930 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.896905    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdd2da80-9a6e-45ee-a107-02516c0540b6-openshift-service-ca\") pod \"maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:38.896988 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.896944    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdd2da80-9a6e-45ee-a107-02516c0540b6-maas-api-tls\") pod \"maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:38.998268 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.998234    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdd2da80-9a6e-45ee-a107-02516c0540b6-openshift-service-ca\") pod \"maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:38.998497 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.998278    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdd2da80-9a6e-45ee-a107-02516c0540b6-maas-api-tls\") pod \"maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:38.998497 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.998312    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f74vs\" (UniqueName: \"kubernetes.io/projected/fdd2da80-9a6e-45ee-a107-02516c0540b6-kube-api-access-f74vs\") pod \"maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:38.998919 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:38.998898    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdd2da80-9a6e-45ee-a107-02516c0540b6-openshift-service-ca\") pod \"maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:39.000934 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:39.000909    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdd2da80-9a6e-45ee-a107-02516c0540b6-maas-api-tls\") pod \"maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:39.006476 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:39.006438    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f74vs\" (UniqueName: \"kubernetes.io/projected/fdd2da80-9a6e-45ee-a107-02516c0540b6-kube-api-access-f74vs\") pod \"maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:39.167874 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:39.167774    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:39.310085 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:39.310057    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"]
Jul 14 16:28:39.311848 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:28:39.311817    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd2da80_9a6e_45ee_a107_02516c0540b6.slice/crio-df67317b46c7043343c146738f04f00714fac4a4d083729098d371bb93896a0d WatchSource:0}: Error finding container df67317b46c7043343c146738f04f00714fac4a4d083729098d371bb93896a0d: Status 404 returned error can't find the container with id df67317b46c7043343c146738f04f00714fac4a4d083729098d371bb93896a0d
Jul 14 16:28:39.619186 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:39.619150    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj" event={"ID":"fdd2da80-9a6e-45ee-a107-02516c0540b6","Type":"ContainerStarted","Data":"df67317b46c7043343c146738f04f00714fac4a4d083729098d371bb93896a0d"}
Jul 14 16:28:40.627011 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:40.626971    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj" event={"ID":"fdd2da80-9a6e-45ee-a107-02516c0540b6","Type":"ContainerStarted","Data":"c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead"}
Jul 14 16:28:40.627404 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:40.627104    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:40.647181 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:40.647126    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj" podStartSLOduration=2.260427426 podStartE2EDuration="2.647109957s" totalImagesPullingTime="386.682531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:28:40.644358027 +0000 UTC m=+2504.163725450" watchObservedRunningTime="2026-07-14 16:28:40.647109957 +0000 UTC m=+2504.166477375"
Jul 14 16:28:46.637344 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:46.637314    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:28:51.622754 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:51.622670    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds"]
Jul 14 16:28:51.626563 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:51.626540    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds"
Jul 14 16:28:51.628979 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:51.628954    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:28:51.635502 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:51.635478    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds"]
Jul 14 16:28:51.713552 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:51.713516    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1f4eba-1c6c-4a79-bb9f-65312909045f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d7e901-r8qds\" (UID: \"1b1f4eba-1c6c-4a79-bb9f-65312909045f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds"
Jul 14 16:28:51.814558 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:51.814526    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1f4eba-1c6c-4a79-bb9f-65312909045f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d7e901-r8qds\" (UID: \"1b1f4eba-1c6c-4a79-bb9f-65312909045f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds"
Jul 14 16:28:51.815134 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:51.815113    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1f4eba-1c6c-4a79-bb9f-65312909045f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d7e901-r8qds\" (UID: \"1b1f4eba-1c6c-4a79-bb9f-65312909045f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds"
Jul 14 16:28:51.938994 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:51.938913    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds"
Jul 14 16:28:52.078460 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:52.078414    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds"]
Jul 14 16:28:52.079818 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:28:52.079793    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b1f4eba_1c6c_4a79_bb9f_65312909045f.slice/crio-854721a1a5e2edebc7918431716d1456d236e1f8ef9bfe0cb5059f33b24abab7 WatchSource:0}: Error finding container 854721a1a5e2edebc7918431716d1456d236e1f8ef9bfe0cb5059f33b24abab7: Status 404 returned error can't find the container with id 854721a1a5e2edebc7918431716d1456d236e1f8ef9bfe0cb5059f33b24abab7
Jul 14 16:28:52.691940 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:52.691899    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds" event={"ID":"1b1f4eba-1c6c-4a79-bb9f-65312909045f","Type":"ContainerStarted","Data":"51ff70c4825360c75ef277c7dcecdfe8a91bc694b6f933657ff6e4d3960b0829"}
Jul 14 16:28:52.692326 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:52.691956    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds" event={"ID":"1b1f4eba-1c6c-4a79-bb9f-65312909045f","Type":"ContainerStarted","Data":"854721a1a5e2edebc7918431716d1456d236e1f8ef9bfe0cb5059f33b24abab7"}
Jul 14 16:28:52.710899 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:52.710853    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds" podStartSLOduration=1.710839483 podStartE2EDuration="1.710839483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:28:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:28:52.709242592 +0000 UTC m=+2516.228610017" watchObservedRunningTime="2026-07-14 16:28:52.710839483 +0000 UTC m=+2516.230206915"
Jul 14 16:28:53.699140 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:53.699110    2560 generic.go:356] "Generic (PLEG): container finished" podID="1b1f4eba-1c6c-4a79-bb9f-65312909045f" containerID="51ff70c4825360c75ef277c7dcecdfe8a91bc694b6f933657ff6e4d3960b0829" exitCode=0
Jul 14 16:28:53.699566 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:53.699199    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds" event={"ID":"1b1f4eba-1c6c-4a79-bb9f-65312909045f","Type":"ContainerDied","Data":"51ff70c4825360c75ef277c7dcecdfe8a91bc694b6f933657ff6e4d3960b0829"}
Jul 14 16:28:54.835088 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:54.835064    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds"
Jul 14 16:28:54.944645 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:54.944609    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1f4eba-1c6c-4a79-bb9f-65312909045f-maas-api-service-ca\") pod \"1b1f4eba-1c6c-4a79-bb9f-65312909045f\" (UID: \"1b1f4eba-1c6c-4a79-bb9f-65312909045f\") "
Jul 14 16:28:54.944972 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:54.944949    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b1f4eba-1c6c-4a79-bb9f-65312909045f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1b1f4eba-1c6c-4a79-bb9f-65312909045f" (UID: "1b1f4eba-1c6c-4a79-bb9f-65312909045f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:28:55.046282 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:55.046253    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1f4eba-1c6c-4a79-bb9f-65312909045f-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:28:55.711638 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:55.711612    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds"
Jul 14 16:28:55.711823 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:55.711655    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7e901-r8qds" event={"ID":"1b1f4eba-1c6c-4a79-bb9f-65312909045f","Type":"ContainerDied","Data":"854721a1a5e2edebc7918431716d1456d236e1f8ef9bfe0cb5059f33b24abab7"}
Jul 14 16:28:55.711823 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:55.711699    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="854721a1a5e2edebc7918431716d1456d236e1f8ef9bfe0cb5059f33b24abab7"
Jul 14 16:28:57.250073 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:28:57.250043    2560 scope.go:117] "RemoveContainer" containerID="52d284c15364a5f0ebaee6dc056bd98b0ece72b246e7631b1a3ef7c20f140c5a"
Jul 14 16:29:02.407983 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.407950    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"]
Jul 14 16:29:02.410651 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.408199    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf" podUID="a7ddc10a-fecc-4820-bcff-0b3641de6b6f" containerName="maas-api" containerID="cri-o://75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c" gracePeriod=30
Jul 14 16:29:02.658103 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.658042    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:29:02.721124 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.721092    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-maas-api-tls\") pod \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") "
Jul 14 16:29:02.721297 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.721132    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xc4f\" (UniqueName: \"kubernetes.io/projected/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-kube-api-access-4xc4f\") pod \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") "
Jul 14 16:29:02.721297 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.721230    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-openshift-service-ca\") pod \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\" (UID: \"a7ddc10a-fecc-4820-bcff-0b3641de6b6f\") "
Jul 14 16:29:02.721626 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.721593    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7ddc10a-fecc-4820-bcff-0b3641de6b6f" (UID: "a7ddc10a-fecc-4820-bcff-0b3641de6b6f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:29:02.723228 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.723198    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7ddc10a-fecc-4820-bcff-0b3641de6b6f" (UID: "a7ddc10a-fecc-4820-bcff-0b3641de6b6f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:29:02.723339 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.723307    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-kube-api-access-4xc4f" (OuterVolumeSpecName: "kube-api-access-4xc4f") pod "a7ddc10a-fecc-4820-bcff-0b3641de6b6f" (UID: "a7ddc10a-fecc-4820-bcff-0b3641de6b6f"). InnerVolumeSpecName "kube-api-access-4xc4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:29:02.749509 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.749480    2560 generic.go:356] "Generic (PLEG): container finished" podID="a7ddc10a-fecc-4820-bcff-0b3641de6b6f" containerID="75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c" exitCode=0
Jul 14 16:29:02.749614 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.749526    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"
Jul 14 16:29:02.749614 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.749548    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf" event={"ID":"a7ddc10a-fecc-4820-bcff-0b3641de6b6f","Type":"ContainerDied","Data":"75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c"}
Jul 14 16:29:02.749614 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.749596    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf" event={"ID":"a7ddc10a-fecc-4820-bcff-0b3641de6b6f","Type":"ContainerDied","Data":"cccd7fd061b24b59bfbaf74994651cad75730369d2b62e29c4a4bd7448995975"}
Jul 14 16:29:02.749719 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.749620    2560 scope.go:117] "RemoveContainer" containerID="75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c"
Jul 14 16:29:02.758900 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.758882    2560 scope.go:117] "RemoveContainer" containerID="75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c"
Jul 14 16:29:02.759139 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:29:02.759121    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c\": container with ID starting with 75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c not found: ID does not exist" containerID="75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c"
Jul 14 16:29:02.759187 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.759151    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c"} err="failed to get container status \"75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c\": rpc error: code = NotFound desc = could not find container \"75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c\": container with ID starting with 75a7c6c500a498915b67a3450f2a0030e8fa90694e6d093dd4ea1cd4c7e0e59c not found: ID does not exist"
Jul 14 16:29:02.771264 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.771238    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"]
Jul 14 16:29:02.775546 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.775526    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7e901-6f849c967d-7rfhf"]
Jul 14 16:29:02.822079 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.822047    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:29:02.822079 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.822072    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:29:02.822079 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:02.822081    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xc4f\" (UniqueName: \"kubernetes.io/projected/a7ddc10a-fecc-4820-bcff-0b3641de6b6f-kube-api-access-4xc4f\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:29:03.036669 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:03.034212    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ddc10a-fecc-4820-bcff-0b3641de6b6f" path="/var/lib/kubelet/pods/a7ddc10a-fecc-4820-bcff-0b3641de6b6f/volumes"
Jul 14 16:29:17.800713 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.800675    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf"]
Jul 14 16:29:17.801281 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.801093    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ddc10a-fecc-4820-bcff-0b3641de6b6f" containerName="maas-api"
Jul 14 16:29:17.801281 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.801104    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ddc10a-fecc-4820-bcff-0b3641de6b6f" containerName="maas-api"
Jul 14 16:29:17.801281 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.801128    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b1f4eba-1c6c-4a79-bb9f-65312909045f" containerName="revoke-keys"
Jul 14 16:29:17.801281 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.801133    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b1f4eba-1c6c-4a79-bb9f-65312909045f" containerName="revoke-keys"
Jul 14 16:29:17.801281 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.801197    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b1f4eba-1c6c-4a79-bb9f-65312909045f" containerName="revoke-keys"
Jul 14 16:29:17.801281 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.801206    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ddc10a-fecc-4820-bcff-0b3641de6b6f" containerName="maas-api"
Jul 14 16:29:17.805762 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.805740    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf"
Jul 14 16:29:17.810577 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.810551    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:29:17.812797 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.812773    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf"]
Jul 14 16:29:17.845536 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.845505    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c226d899-1aea-4db8-8b25-8c6ed7b8fce4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c40588-5gcdf\" (UID: \"c226d899-1aea-4db8-8b25-8c6ed7b8fce4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf"
Jul 14 16:29:17.946721 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.946682    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c226d899-1aea-4db8-8b25-8c6ed7b8fce4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c40588-5gcdf\" (UID: \"c226d899-1aea-4db8-8b25-8c6ed7b8fce4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf"
Jul 14 16:29:17.947349 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:17.947331    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c226d899-1aea-4db8-8b25-8c6ed7b8fce4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c40588-5gcdf\" (UID: \"c226d899-1aea-4db8-8b25-8c6ed7b8fce4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf"
Jul 14 16:29:18.118507 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:18.118474    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf"
Jul 14 16:29:18.252170 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:18.252132    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf"]
Jul 14 16:29:18.253628 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:29:18.253599    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc226d899_1aea_4db8_8b25_8c6ed7b8fce4.slice/crio-c6974299b5e819e6be479f3b6e316d7ee3910f00546b15822cbe19a5f9df1c85 WatchSource:0}: Error finding container c6974299b5e819e6be479f3b6e316d7ee3910f00546b15822cbe19a5f9df1c85: Status 404 returned error can't find the container with id c6974299b5e819e6be479f3b6e316d7ee3910f00546b15822cbe19a5f9df1c85
Jul 14 16:29:18.835179 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:18.835079    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf" event={"ID":"c226d899-1aea-4db8-8b25-8c6ed7b8fce4","Type":"ContainerStarted","Data":"27241252f7ad5a80e0b834c475ff5250aff87f57f8332af2786036b69795710e"}
Jul 14 16:29:18.835179 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:18.835131    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf" event={"ID":"c226d899-1aea-4db8-8b25-8c6ed7b8fce4","Type":"ContainerStarted","Data":"c6974299b5e819e6be479f3b6e316d7ee3910f00546b15822cbe19a5f9df1c85"}
Jul 14 16:29:18.857676 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:18.857625    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf" podStartSLOduration=1.857604652 podStartE2EDuration="1.857604652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:29:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:29:18.857186357 +0000 UTC m=+2542.376553779" watchObservedRunningTime="2026-07-14 16:29:18.857604652 +0000 UTC m=+2542.376972074"
Jul 14 16:29:19.842919 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:19.842883    2560 generic.go:356] "Generic (PLEG): container finished" podID="c226d899-1aea-4db8-8b25-8c6ed7b8fce4" containerID="27241252f7ad5a80e0b834c475ff5250aff87f57f8332af2786036b69795710e" exitCode=0
Jul 14 16:29:19.843345 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:19.842971    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf" event={"ID":"c226d899-1aea-4db8-8b25-8c6ed7b8fce4","Type":"ContainerDied","Data":"27241252f7ad5a80e0b834c475ff5250aff87f57f8332af2786036b69795710e"}
Jul 14 16:29:20.978109 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:20.978087    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf"
Jul 14 16:29:21.077204 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:21.077175    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c226d899-1aea-4db8-8b25-8c6ed7b8fce4-maas-api-service-ca\") pod \"c226d899-1aea-4db8-8b25-8c6ed7b8fce4\" (UID: \"c226d899-1aea-4db8-8b25-8c6ed7b8fce4\") "
Jul 14 16:29:21.077523 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:21.077503    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c226d899-1aea-4db8-8b25-8c6ed7b8fce4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c226d899-1aea-4db8-8b25-8c6ed7b8fce4" (UID: "c226d899-1aea-4db8-8b25-8c6ed7b8fce4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:29:21.178007 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:21.177928    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c226d899-1aea-4db8-8b25-8c6ed7b8fce4-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:29:21.854852 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:21.854821    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf"
Jul 14 16:29:21.855013 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:21.854826    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c40588-5gcdf" event={"ID":"c226d899-1aea-4db8-8b25-8c6ed7b8fce4","Type":"ContainerDied","Data":"c6974299b5e819e6be479f3b6e316d7ee3910f00546b15822cbe19a5f9df1c85"}
Jul 14 16:29:21.855013 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:21.854941    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6974299b5e819e6be479f3b6e316d7ee3910f00546b15822cbe19a5f9df1c85"
Jul 14 16:29:28.312320 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.312278    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"]
Jul 14 16:29:28.313248 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.313191    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj" podUID="fdd2da80-9a6e-45ee-a107-02516c0540b6" containerName="maas-api" containerID="cri-o://c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead" gracePeriod=30
Jul 14 16:29:28.563095 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.563037    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:29:28.648034 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.648004    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdd2da80-9a6e-45ee-a107-02516c0540b6-openshift-service-ca\") pod \"fdd2da80-9a6e-45ee-a107-02516c0540b6\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") "
Jul 14 16:29:28.648186 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.648043    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdd2da80-9a6e-45ee-a107-02516c0540b6-maas-api-tls\") pod \"fdd2da80-9a6e-45ee-a107-02516c0540b6\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") "
Jul 14 16:29:28.648186 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.648101    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f74vs\" (UniqueName: \"kubernetes.io/projected/fdd2da80-9a6e-45ee-a107-02516c0540b6-kube-api-access-f74vs\") pod \"fdd2da80-9a6e-45ee-a107-02516c0540b6\" (UID: \"fdd2da80-9a6e-45ee-a107-02516c0540b6\") "
Jul 14 16:29:28.648407 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.648385    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdd2da80-9a6e-45ee-a107-02516c0540b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fdd2da80-9a6e-45ee-a107-02516c0540b6" (UID: "fdd2da80-9a6e-45ee-a107-02516c0540b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:29:28.650270 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.650245    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdd2da80-9a6e-45ee-a107-02516c0540b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fdd2da80-9a6e-45ee-a107-02516c0540b6" (UID: "fdd2da80-9a6e-45ee-a107-02516c0540b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:29:28.650358 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.650279    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdd2da80-9a6e-45ee-a107-02516c0540b6-kube-api-access-f74vs" (OuterVolumeSpecName: "kube-api-access-f74vs") pod "fdd2da80-9a6e-45ee-a107-02516c0540b6" (UID: "fdd2da80-9a6e-45ee-a107-02516c0540b6"). InnerVolumeSpecName "kube-api-access-f74vs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:29:28.748969 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.748939    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdd2da80-9a6e-45ee-a107-02516c0540b6-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:29:28.748969 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.748966    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdd2da80-9a6e-45ee-a107-02516c0540b6-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:29:28.748969 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.748974    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f74vs\" (UniqueName: \"kubernetes.io/projected/fdd2da80-9a6e-45ee-a107-02516c0540b6-kube-api-access-f74vs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:29:28.893003 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.892974    2560 generic.go:356] "Generic (PLEG): container finished" podID="fdd2da80-9a6e-45ee-a107-02516c0540b6" containerID="c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead" exitCode=0
Jul 14 16:29:28.893143 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.893028    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"
Jul 14 16:29:28.893143 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.893059    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj" event={"ID":"fdd2da80-9a6e-45ee-a107-02516c0540b6","Type":"ContainerDied","Data":"c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead"}
Jul 14 16:29:28.893143 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.893118    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj" event={"ID":"fdd2da80-9a6e-45ee-a107-02516c0540b6","Type":"ContainerDied","Data":"df67317b46c7043343c146738f04f00714fac4a4d083729098d371bb93896a0d"}
Jul 14 16:29:28.893333 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.893148    2560 scope.go:117] "RemoveContainer" containerID="c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead"
Jul 14 16:29:28.902201 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.902184    2560 scope.go:117] "RemoveContainer" containerID="c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead"
Jul 14 16:29:28.902486 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:29:28.902462    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead\": container with ID starting with c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead not found: ID does not exist" containerID="c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead"
Jul 14 16:29:28.902585 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.902492    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead"} err="failed to get container status \"c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead\": rpc error: code = NotFound desc = could not find container \"c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead\": container with ID starting with c32fc0560fd32b3fb95b4bc723e2dd8e0bc4d0af1c56385514737fb63804aead not found: ID does not exist"
Jul 14 16:29:28.913976 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.913956    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"]
Jul 14 16:29:28.919377 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:28.919353    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c40588-d4bf857bf-jnqtj"]
Jul 14 16:29:29.032893 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:29.032860    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdd2da80-9a6e-45ee-a107-02516c0540b6" path="/var/lib/kubelet/pods/fdd2da80-9a6e-45ee-a107-02516c0540b6/volumes"
Jul 14 16:29:48.172354 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.172319    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"]
Jul 14 16:29:48.172996 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.172975    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c226d899-1aea-4db8-8b25-8c6ed7b8fce4" containerName="revoke-keys"
Jul 14 16:29:48.173051 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.173001    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c226d899-1aea-4db8-8b25-8c6ed7b8fce4" containerName="revoke-keys"
Jul 14 16:29:48.173051 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.173023    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdd2da80-9a6e-45ee-a107-02516c0540b6" containerName="maas-api"
Jul 14 16:29:48.173051 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.173032    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdd2da80-9a6e-45ee-a107-02516c0540b6" containerName="maas-api"
Jul 14 16:29:48.173176 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.173162    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdd2da80-9a6e-45ee-a107-02516c0540b6" containerName="maas-api"
Jul 14 16:29:48.173211 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.173179    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c226d899-1aea-4db8-8b25-8c6ed7b8fce4" containerName="revoke-keys"
Jul 14 16:29:48.177072 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.177051    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:48.180710 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.180688    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 16:29:48.180886 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.180862    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5kg5z\""
Jul 14 16:29:48.181020 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.180792    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-21b35e-serving-cert\""
Jul 14 16:29:48.181020 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.180841    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 16:29:48.191769 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.191747    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"]
Jul 14 16:29:48.327553 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.327511    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28835b46-101f-4c25-82af-711c6d1d9419-maas-api-tls\") pod \"maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:48.327733 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.327558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28835b46-101f-4c25-82af-711c6d1d9419-openshift-service-ca\") pod \"maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:48.327733 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.327617    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t87hs\" (UniqueName: \"kubernetes.io/projected/28835b46-101f-4c25-82af-711c6d1d9419-kube-api-access-t87hs\") pod \"maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:48.428926 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.428824    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28835b46-101f-4c25-82af-711c6d1d9419-maas-api-tls\") pod \"maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:48.428926 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.428880    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28835b46-101f-4c25-82af-711c6d1d9419-openshift-service-ca\") pod \"maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:48.429167 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.428928    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t87hs\" (UniqueName: \"kubernetes.io/projected/28835b46-101f-4c25-82af-711c6d1d9419-kube-api-access-t87hs\") pod \"maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:48.429565 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.429537    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28835b46-101f-4c25-82af-711c6d1d9419-openshift-service-ca\") pod \"maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:48.431341 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.431320    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28835b46-101f-4c25-82af-711c6d1d9419-maas-api-tls\") pod \"maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:48.446930 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.446901    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t87hs\" (UniqueName: \"kubernetes.io/projected/28835b46-101f-4c25-82af-711c6d1d9419-kube-api-access-t87hs\") pod \"maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:48.487848 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.487810    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:48.632083 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:48.632057    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"]
Jul 14 16:29:48.633867 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:29:48.633840    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28835b46_101f_4c25_82af_711c6d1d9419.slice/crio-f4c6b23b49d51c43132c9ebd2c088b264817f596f222751bbcea7b88ebd97c5b WatchSource:0}: Error finding container f4c6b23b49d51c43132c9ebd2c088b264817f596f222751bbcea7b88ebd97c5b: Status 404 returned error can't find the container with id f4c6b23b49d51c43132c9ebd2c088b264817f596f222751bbcea7b88ebd97c5b
Jul 14 16:29:49.000579 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:49.000547    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s" event={"ID":"28835b46-101f-4c25-82af-711c6d1d9419","Type":"ContainerStarted","Data":"f4c6b23b49d51c43132c9ebd2c088b264817f596f222751bbcea7b88ebd97c5b"}
Jul 14 16:29:51.012892 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:51.012858    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s" event={"ID":"28835b46-101f-4c25-82af-711c6d1d9419","Type":"ContainerStarted","Data":"9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98"}
Jul 14 16:29:51.013323 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:51.012922    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:51.037053 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:51.037012    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s" podStartSLOduration=1.166384506 podStartE2EDuration="3.036996281s" totalImagesPullingTime="1.870611775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:29:51.036280912 +0000 UTC m=+2574.555648332" watchObservedRunningTime="2026-07-14 16:29:51.036996281 +0000 UTC m=+2574.556363698"
Jul 14 16:29:57.023343 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.023315    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:29:57.305280 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.305252    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"]
Jul 14 16:29:57.310646 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.310627    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:57.314349 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.314333    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-4e27c0-serving-cert\""
Jul 14 16:29:57.327212 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.327189    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"]
Jul 14 16:29:57.407604 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.407568    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh5vg\" (UniqueName: \"kubernetes.io/projected/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-kube-api-access-vh5vg\") pod \"maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:57.407832 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.407634    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:57.407832 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.407669    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-maas-api-tls\") pod \"maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:57.508822 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.508787    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh5vg\" (UniqueName: \"kubernetes.io/projected/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-kube-api-access-vh5vg\") pod \"maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:57.509000 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.508857    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:57.509000 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.508889    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-maas-api-tls\") pod \"maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:57.509550 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.509525    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:57.511597 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.511577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-maas-api-tls\") pod \"maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:57.519812 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.519792    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh5vg\" (UniqueName: \"kubernetes.io/projected/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-kube-api-access-vh5vg\") pod \"maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:57.620847 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.620820    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:57.761793 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:57.761761    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"]
Jul 14 16:29:57.762904 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:29:57.762879    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e8dbf7a_6d96_48f7_88e5_fdaae81d29b5.slice/crio-6951205fab00cdc086e3d3abebe0441f7db586d8315d01fe83fd69504a511750 WatchSource:0}: Error finding container 6951205fab00cdc086e3d3abebe0441f7db586d8315d01fe83fd69504a511750: Status 404 returned error can't find the container with id 6951205fab00cdc086e3d3abebe0441f7db586d8315d01fe83fd69504a511750
Jul 14 16:29:58.049650 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:58.049561    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn" event={"ID":"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5","Type":"ContainerStarted","Data":"6951205fab00cdc086e3d3abebe0441f7db586d8315d01fe83fd69504a511750"}
Jul 14 16:29:59.056110 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:59.056070    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn" event={"ID":"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5","Type":"ContainerStarted","Data":"e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25"}
Jul 14 16:29:59.056567 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:59.056201    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:29:59.092273 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:29:59.092225    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn" podStartSLOduration=1.741778065 podStartE2EDuration="2.092209931s" totalImagesPullingTime="350.431866ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:29:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:29:59.091669342 +0000 UTC m=+2582.611036762" watchObservedRunningTime="2026-07-14 16:29:59.092209931 +0000 UTC m=+2582.611577352"
Jul 14 16:30:00.152360 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.152322    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg"]
Jul 14 16:30:00.156431 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.156405    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg"
Jul 14 16:30:00.156603 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.156439    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr"]
Jul 14 16:30:00.159376 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.159357    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:30:00.160599 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.160584    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw"]
Jul 14 16:30:00.160719 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.160700    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr"
Jul 14 16:30:00.164150 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.164042    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw"
Jul 14 16:30:00.168123 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.168107    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg"
Jul 14 16:30:00.175613 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.175594    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr"
Jul 14 16:30:00.178255 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.178233    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg"]
Jul 14 16:30:00.181372 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.181291    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr"]
Jul 14 16:30:00.185402 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.185380    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw"]
Jul 14 16:30:00.213173 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.213147    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw"
Jul 14 16:30:00.358803 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.358771    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg"]
Jul 14 16:30:00.360181 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:30:00.360147    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4516f11f_7b0a_42e7_ae40_f945b83d58fd.slice/crio-d50704e3cc99d1b9f1e0be7f76eb827c99deeecb4f61bc6cb1f07426c9517bf7 WatchSource:0}: Error finding container d50704e3cc99d1b9f1e0be7f76eb827c99deeecb4f61bc6cb1f07426c9517bf7: Status 404 returned error can't find the container with id d50704e3cc99d1b9f1e0be7f76eb827c99deeecb4f61bc6cb1f07426c9517bf7
Jul 14 16:30:00.376234 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.376204    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr"]
Jul 14 16:30:00.378161 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:30:00.378131    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42b0809_ca9d_49a2_862d_9812444e6483.slice/crio-36fee18835d77b4abbe6d035b658fe0912aecdbd9865ce033bf400283eb95a08 WatchSource:0}: Error finding container 36fee18835d77b4abbe6d035b658fe0912aecdbd9865ce033bf400283eb95a08: Status 404 returned error can't find the container with id 36fee18835d77b4abbe6d035b658fe0912aecdbd9865ce033bf400283eb95a08
Jul 14 16:30:00.414434 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:00.414382    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw"]
Jul 14 16:30:00.417379 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:30:00.417352    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ea790f_efbe_4a06_8130_0a5d98dd7fc1.slice/crio-6cd22ab212296ba4405357a72af3adf18cc48998be02c4e792d56edbff3618ad WatchSource:0}: Error finding container 6cd22ab212296ba4405357a72af3adf18cc48998be02c4e792d56edbff3618ad: Status 404 returned error can't find the container with id 6cd22ab212296ba4405357a72af3adf18cc48998be02c4e792d56edbff3618ad
Jul 14 16:30:01.068073 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:01.067976    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr" event={"ID":"f42b0809-ca9d-49a2-862d-9812444e6483","Type":"ContainerStarted","Data":"ccacac42339f64582133b8713e9afa4171f3a2c54ab4e949aadbdd0404e690a9"}
Jul 14 16:30:01.068073 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:01.068037    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr" event={"ID":"f42b0809-ca9d-49a2-862d-9812444e6483","Type":"ContainerStarted","Data":"36fee18835d77b4abbe6d035b658fe0912aecdbd9865ce033bf400283eb95a08"}
Jul 14 16:30:01.069492 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:01.069468    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw" event={"ID":"37ea790f-efbe-4a06-8130-0a5d98dd7fc1","Type":"ContainerStarted","Data":"24c10bd7edd5698953a05f89b4154ec740a4540551b0aeffd096429f9a0e5007"}
Jul 14 16:30:01.069613 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:01.069514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw" event={"ID":"37ea790f-efbe-4a06-8130-0a5d98dd7fc1","Type":"ContainerStarted","Data":"6cd22ab212296ba4405357a72af3adf18cc48998be02c4e792d56edbff3618ad"}
Jul 14 16:30:01.070870 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:01.070846    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg" event={"ID":"4516f11f-7b0a-42e7-ae40-f945b83d58fd","Type":"ContainerStarted","Data":"dcc7a62ffbc385d87d2d291347aa21286a4a1a91fa8821e0c58c0bc4da86e71a"}
Jul 14 16:30:01.070986 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:01.070884    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg" event={"ID":"4516f11f-7b0a-42e7-ae40-f945b83d58fd","Type":"ContainerStarted","Data":"d50704e3cc99d1b9f1e0be7f76eb827c99deeecb4f61bc6cb1f07426c9517bf7"}
Jul 14 16:30:01.085243 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:01.085198    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr" podStartSLOduration=1.085185004 podStartE2EDuration="1.085185004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:30:01.084336127 +0000 UTC m=+2584.603703548" watchObservedRunningTime="2026-07-14 16:30:01.085185004 +0000 UTC m=+2584.604552494"
Jul 14 16:30:01.103209 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:01.103163    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw" podStartSLOduration=1.103148627 podStartE2EDuration="1.103148627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:30:01.100347376 +0000 UTC m=+2584.619714797" watchObservedRunningTime="2026-07-14 16:30:01.103148627 +0000 UTC m=+2584.622516047"
Jul 14 16:30:01.118432 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:01.118389    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg" podStartSLOduration=1.118374904 podStartE2EDuration="1.118374904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:30:01.115919329 +0000 UTC m=+2584.635286751" watchObservedRunningTime="2026-07-14 16:30:01.118374904 +0000 UTC m=+2584.637742341"
Jul 14 16:30:02.078796 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:02.078764    2560 generic.go:356] "Generic (PLEG): container finished" podID="f42b0809-ca9d-49a2-862d-9812444e6483" containerID="ccacac42339f64582133b8713e9afa4171f3a2c54ab4e949aadbdd0404e690a9" exitCode=0
Jul 14 16:30:02.079262 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:02.078850    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr" event={"ID":"f42b0809-ca9d-49a2-862d-9812444e6483","Type":"ContainerDied","Data":"ccacac42339f64582133b8713e9afa4171f3a2c54ab4e949aadbdd0404e690a9"}
Jul 14 16:30:02.087361 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:02.087334    2560 generic.go:356] "Generic (PLEG): container finished" podID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerID="24c10bd7edd5698953a05f89b4154ec740a4540551b0aeffd096429f9a0e5007" exitCode=7
Jul 14 16:30:02.087551 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:02.087416    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw" event={"ID":"37ea790f-efbe-4a06-8130-0a5d98dd7fc1","Type":"ContainerDied","Data":"24c10bd7edd5698953a05f89b4154ec740a4540551b0aeffd096429f9a0e5007"}
Jul 14 16:30:02.087551 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:02.087544    2560 scope.go:117] "RemoveContainer" containerID="24c10bd7edd5698953a05f89b4154ec740a4540551b0aeffd096429f9a0e5007"
Jul 14 16:30:02.089067 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:02.089043    2560 generic.go:356] "Generic (PLEG): container finished" podID="4516f11f-7b0a-42e7-ae40-f945b83d58fd" containerID="dcc7a62ffbc385d87d2d291347aa21286a4a1a91fa8821e0c58c0bc4da86e71a" exitCode=0
Jul 14 16:30:02.089277 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:02.089125    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg" event={"ID":"4516f11f-7b0a-42e7-ae40-f945b83d58fd","Type":"ContainerDied","Data":"dcc7a62ffbc385d87d2d291347aa21286a4a1a91fa8821e0c58c0bc4da86e71a"}
Jul 14 16:30:03.097054 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:03.097021    2560 generic.go:356] "Generic (PLEG): container finished" podID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerID="48916803725e3b33935657973895a85eed9f9b932025055b90b789d89a4201ec" exitCode=7
Jul 14 16:30:03.097515 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:03.097105    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw" event={"ID":"37ea790f-efbe-4a06-8130-0a5d98dd7fc1","Type":"ContainerDied","Data":"48916803725e3b33935657973895a85eed9f9b932025055b90b789d89a4201ec"}
Jul 14 16:30:03.097515 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:03.097162    2560 scope.go:117] "RemoveContainer" containerID="24c10bd7edd5698953a05f89b4154ec740a4540551b0aeffd096429f9a0e5007"
Jul 14 16:30:03.097515 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:03.097339    2560 scope.go:117] "RemoveContainer" containerID="48916803725e3b33935657973895a85eed9f9b932025055b90b789d89a4201ec"
Jul 14 16:30:03.097969 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:30:03.097644    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw_odh-ai-gateway-infra(37ea790f-efbe-4a06-8130-0a5d98dd7fc1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1"
Jul 14 16:30:03.242064 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:03.242036    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg"
Jul 14 16:30:03.259646 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:03.259621    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr"
Jul 14 16:30:04.104125 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:04.104093    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg"
Jul 14 16:30:04.104591 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:04.104132    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg" event={"ID":"4516f11f-7b0a-42e7-ae40-f945b83d58fd","Type":"ContainerDied","Data":"d50704e3cc99d1b9f1e0be7f76eb827c99deeecb4f61bc6cb1f07426c9517bf7"}
Jul 14 16:30:04.104591 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:04.104188    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d50704e3cc99d1b9f1e0be7f76eb827c99deeecb4f61bc6cb1f07426c9517bf7"
Jul 14 16:30:04.105556 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:04.105532    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr"
Jul 14 16:30:04.105686 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:04.105537    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734110-vw2mr" event={"ID":"f42b0809-ca9d-49a2-862d-9812444e6483","Type":"ContainerDied","Data":"36fee18835d77b4abbe6d035b658fe0912aecdbd9865ce033bf400283eb95a08"}
Jul 14 16:30:04.105686 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:04.105648    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36fee18835d77b4abbe6d035b658fe0912aecdbd9865ce033bf400283eb95a08"
Jul 14 16:30:04.107342 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:04.107322    2560 scope.go:117] "RemoveContainer" containerID="48916803725e3b33935657973895a85eed9f9b932025055b90b789d89a4201ec"
Jul 14 16:30:04.107602 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:30:04.107580    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw_odh-ai-gateway-infra(37ea790f-efbe-4a06-8130-0a5d98dd7fc1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1"
Jul 14 16:30:05.066946 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:05.066917    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:30:15.117202 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.117167    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"]
Jul 14 16:30:15.117638 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.117623    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4516f11f-7b0a-42e7-ae40-f945b83d58fd" containerName="cleanup"
Jul 14 16:30:15.117703 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.117642    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4516f11f-7b0a-42e7-ae40-f945b83d58fd" containerName="cleanup"
Jul 14 16:30:15.117703 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.117656    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f42b0809-ca9d-49a2-862d-9812444e6483" containerName="cleanup"
Jul 14 16:30:15.117703 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.117662    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f42b0809-ca9d-49a2-862d-9812444e6483" containerName="cleanup"
Jul 14 16:30:15.117821 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.117755    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f42b0809-ca9d-49a2-862d-9812444e6483" containerName="cleanup"
Jul 14 16:30:15.117821 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.117770    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4516f11f-7b0a-42e7-ae40-f945b83d58fd" containerName="cleanup"
Jul 14 16:30:15.121148 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.121129    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.125336 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.125317    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"kube-root-ca.crt\""
Jul 14 16:30:15.125792 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.125777    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"auth-test-model-21b35e-kserve-self-signed-certs\""
Jul 14 16:30:15.125859 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.125777    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"default-dockercfg-sf92j\""
Jul 14 16:30:15.125859 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.125843    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"openshift-service-ca.crt\""
Jul 14 16:30:15.132303 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.132280    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"]
Jul 14 16:30:15.185256 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.185224    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-model-cache\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.185426 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.185282    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-tmp-dir\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.185426 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.185327    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6fv5\" (UniqueName: \"kubernetes.io/projected/a3d42637-3c0c-4508-a246-c634a54898a6-kube-api-access-s6fv5\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.185426 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.185417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d42637-3c0c-4508-a246-c634a54898a6-tls-certs\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.185594 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.185552    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-dshm\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.185594 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.185584    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-home\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.286945 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.286903    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-tmp-dir\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.287109 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.286953    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6fv5\" (UniqueName: \"kubernetes.io/projected/a3d42637-3c0c-4508-a246-c634a54898a6-kube-api-access-s6fv5\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.287109 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.287014    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d42637-3c0c-4508-a246-c634a54898a6-tls-certs\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.287109 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.287068    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-dshm\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.287109 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.287093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-home\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.287334 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.287172    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-model-cache\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.287395 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.287328    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-tmp-dir\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.287487 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.287466    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-home\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.287551 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.287534    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-model-cache\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.289436 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.289416    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-dshm\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.289741 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.289721    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d42637-3c0c-4508-a246-c634a54898a6-tls-certs\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.295217 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.295192    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6fv5\" (UniqueName: \"kubernetes.io/projected/a3d42637-3c0c-4508-a246-c634a54898a6-kube-api-access-s6fv5\") pod \"auth-test-model-21b35e-kserve-6b684dc74c-kkkhm\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") " pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.431113 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.431009    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:15.575973 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:15.575939    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"]
Jul 14 16:30:15.577330 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:30:15.577305    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3d42637_3c0c_4508_a246_c634a54898a6.slice/crio-6ab3561b40707414672423c299b2a87cbe8b325a023b6c50c6599ff74bee7066 WatchSource:0}: Error finding container 6ab3561b40707414672423c299b2a87cbe8b325a023b6c50c6599ff74bee7066: Status 404 returned error can't find the container with id 6ab3561b40707414672423c299b2a87cbe8b325a023b6c50c6599ff74bee7066
Jul 14 16:30:16.026181 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:16.026150    2560 scope.go:117] "RemoveContainer" containerID="48916803725e3b33935657973895a85eed9f9b932025055b90b789d89a4201ec"
Jul 14 16:30:16.172837 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:16.172799    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm" event={"ID":"a3d42637-3c0c-4508-a246-c634a54898a6","Type":"ContainerStarted","Data":"da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa"}
Jul 14 16:30:16.173272 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:16.172849    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm" event={"ID":"a3d42637-3c0c-4508-a246-c634a54898a6","Type":"ContainerStarted","Data":"6ab3561b40707414672423c299b2a87cbe8b325a023b6c50c6599ff74bee7066"}
Jul 14 16:30:16.194295 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:16.194242    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm" podStartSLOduration=1.194223942 podStartE2EDuration="1.194223942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:30:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:30:16.192316655 +0000 UTC m=+2599.711684078" watchObservedRunningTime="2026-07-14 16:30:16.194223942 +0000 UTC m=+2599.713591364"
Jul 14 16:30:17.180153 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:17.180118    2560 generic.go:356] "Generic (PLEG): container finished" podID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerID="935fc39369d5d8ee2ac998c323bf2445899e6c986f187695b1cf1be0166cc93c" exitCode=0
Jul 14 16:30:17.180627 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:17.180192    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw" event={"ID":"37ea790f-efbe-4a06-8130-0a5d98dd7fc1","Type":"ContainerDied","Data":"935fc39369d5d8ee2ac998c323bf2445899e6c986f187695b1cf1be0166cc93c"}
Jul 14 16:30:17.180627 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:17.180252    2560 scope.go:117] "RemoveContainer" containerID="48916803725e3b33935657973895a85eed9f9b932025055b90b789d89a4201ec"
Jul 14 16:30:18.247192 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:18.247161    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw"]
Jul 14 16:30:18.323613 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:18.323590    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw"
Jul 14 16:30:19.200558 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:19.200514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw" event={"ID":"37ea790f-efbe-4a06-8130-0a5d98dd7fc1","Type":"ContainerDied","Data":"6cd22ab212296ba4405357a72af3adf18cc48998be02c4e792d56edbff3618ad"}
Jul 14 16:30:19.200757 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:19.200568    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw"
Jul 14 16:30:19.200757 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:19.200611    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6cd22ab212296ba4405357a72af3adf18cc48998be02c4e792d56edbff3618ad"
Jul 14 16:30:19.234485 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:19.234440    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw"]
Jul 14 16:30:19.239136 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:19.239113    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4e27c0-29734110-p4xqw"]
Jul 14 16:30:21.032628 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:21.032597    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" path="/var/lib/kubelet/pods/37ea790f-efbe-4a06-8130-0a5d98dd7fc1/volumes"
Jul 14 16:30:25.431632 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:25.431594    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:25.432020 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:25.431649    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:25.444413 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:25.444391    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:26.250569 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:26.250543    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:30:34.203061 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.203018    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"]
Jul 14 16:30:34.203735 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.203713    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerName="cleanup"
Jul 14 16:30:34.203788 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.203740    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerName="cleanup"
Jul 14 16:30:34.203788 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.203761    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerName="cleanup"
Jul 14 16:30:34.203788 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.203769    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerName="cleanup"
Jul 14 16:30:34.203900 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.203888    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerName="cleanup"
Jul 14 16:30:34.203936 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.203906    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerName="cleanup"
Jul 14 16:30:34.207607 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.207583    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.211055 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.211026    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"openshift-service-ca.crt\""
Jul 14 16:30:34.211285 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.211184    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"kube-root-ca.crt\""
Jul 14 16:30:34.211285 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.211198    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"default-dockercfg-78952\""
Jul 14 16:30:34.211468 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.211180    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"auth-test-model-4e27c0-kserve-self-signed-certs\""
Jul 14 16:30:34.214566 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.214544    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"]
Jul 14 16:30:34.270900 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.270872    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-tmp-dir\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.271021 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.270905    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c711639-401d-429e-acb0-cf79bf614b11-tls-certs\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.271021 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.270939    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-home\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.271021 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.271006    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6g4d\" (UniqueName: \"kubernetes.io/projected/4c711639-401d-429e-acb0-cf79bf614b11-kube-api-access-q6g4d\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.271143 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.271042    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-dshm\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.271143 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.271057    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-model-cache\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.371679 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.371645    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6g4d\" (UniqueName: \"kubernetes.io/projected/4c711639-401d-429e-acb0-cf79bf614b11-kube-api-access-q6g4d\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.371856 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.371707    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-dshm\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.371856 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.371725    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-model-cache\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.371856 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.371751    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-tmp-dir\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.371856 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.371775    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c711639-401d-429e-acb0-cf79bf614b11-tls-certs\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.371856 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.371794    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-home\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.372176 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.372155    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-model-cache\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.372238 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.372216    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-tmp-dir\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.372238 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.372231    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-home\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.373822 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.373804    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-dshm\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.374328 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.374310    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c711639-401d-429e-acb0-cf79bf614b11-tls-certs\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.381627 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.381606    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6g4d\" (UniqueName: \"kubernetes.io/projected/4c711639-401d-429e-acb0-cf79bf614b11-kube-api-access-q6g4d\") pod \"auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") " pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.517820 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.517731    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:34.654068 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:34.654043    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"]
Jul 14 16:30:34.655611 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:30:34.655577    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c711639_401d_429e_acb0_cf79bf614b11.slice/crio-bf57073a26f822d275e2d21f083bbccdc6e6ce3a8d20317fae09b78076073668 WatchSource:0}: Error finding container bf57073a26f822d275e2d21f083bbccdc6e6ce3a8d20317fae09b78076073668: Status 404 returned error can't find the container with id bf57073a26f822d275e2d21f083bbccdc6e6ce3a8d20317fae09b78076073668
Jul 14 16:30:35.288422 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:35.288388    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb" event={"ID":"4c711639-401d-429e-acb0-cf79bf614b11","Type":"ContainerStarted","Data":"7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6"}
Jul 14 16:30:35.288845 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:35.288429    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb" event={"ID":"4c711639-401d-429e-acb0-cf79bf614b11","Type":"ContainerStarted","Data":"bf57073a26f822d275e2d21f083bbccdc6e6ce3a8d20317fae09b78076073668"}
Jul 14 16:30:35.308607 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:35.308549    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb" podStartSLOduration=1.308530102 podStartE2EDuration="1.308530102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:30:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:30:35.308017138 +0000 UTC m=+2618.827384560" watchObservedRunningTime="2026-07-14 16:30:35.308530102 +0000 UTC m=+2618.827897523"
Jul 14 16:30:44.518819 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:44.518779    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:44.518819 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:44.518830    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:44.530993 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:44.530966    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:30:45.353471 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:30:45.353425    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:31:57.151921 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:31:57.151845    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:31:57.166327 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:31:57.166303    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:32:45.010235 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:32:45.010197    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"]
Jul 14 16:32:45.799164 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:32:45.799130    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"]
Jul 14 16:33:00.030760 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.030690    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm" podUID="a3d42637-3c0c-4508-a246-c634a54898a6" containerName="main" containerID="cri-o://da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa" gracePeriod=45
Jul 14 16:33:00.292778 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.292714    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:33:00.349549 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.349516    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-tmp-dir\") pod \"a3d42637-3c0c-4508-a246-c634a54898a6\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") "
Jul 14 16:33:00.349549 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.349552    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-home\") pod \"a3d42637-3c0c-4508-a246-c634a54898a6\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") "
Jul 14 16:33:00.349769 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.349646    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d42637-3c0c-4508-a246-c634a54898a6-tls-certs\") pod \"a3d42637-3c0c-4508-a246-c634a54898a6\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") "
Jul 14 16:33:00.349769 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.349693    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-model-cache\") pod \"a3d42637-3c0c-4508-a246-c634a54898a6\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") "
Jul 14 16:33:00.349769 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.349713    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6fv5\" (UniqueName: \"kubernetes.io/projected/a3d42637-3c0c-4508-a246-c634a54898a6-kube-api-access-s6fv5\") pod \"a3d42637-3c0c-4508-a246-c634a54898a6\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") "
Jul 14 16:33:00.349769 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.349747    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-dshm\") pod \"a3d42637-3c0c-4508-a246-c634a54898a6\" (UID: \"a3d42637-3c0c-4508-a246-c634a54898a6\") "
Jul 14 16:33:00.349966 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.349779    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a3d42637-3c0c-4508-a246-c634a54898a6" (UID: "a3d42637-3c0c-4508-a246-c634a54898a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:33:00.351499 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.350155    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-home" (OuterVolumeSpecName: "home") pod "a3d42637-3c0c-4508-a246-c634a54898a6" (UID: "a3d42637-3c0c-4508-a246-c634a54898a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:33:00.351499 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.350385    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-model-cache" (OuterVolumeSpecName: "model-cache") pod "a3d42637-3c0c-4508-a246-c634a54898a6" (UID: "a3d42637-3c0c-4508-a246-c634a54898a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:33:00.351499 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.351170    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-tmp-dir\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:00.352957 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.352924    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3d42637-3c0c-4508-a246-c634a54898a6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a3d42637-3c0c-4508-a246-c634a54898a6" (UID: "a3d42637-3c0c-4508-a246-c634a54898a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:33:00.353529 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.353485    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d42637-3c0c-4508-a246-c634a54898a6-kube-api-access-s6fv5" (OuterVolumeSpecName: "kube-api-access-s6fv5") pod "a3d42637-3c0c-4508-a246-c634a54898a6" (UID: "a3d42637-3c0c-4508-a246-c634a54898a6"). InnerVolumeSpecName "kube-api-access-s6fv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:33:00.359157 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.359124    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-dshm" (OuterVolumeSpecName: "dshm") pod "a3d42637-3c0c-4508-a246-c634a54898a6" (UID: "a3d42637-3c0c-4508-a246-c634a54898a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:33:00.451935 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.451898    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d42637-3c0c-4508-a246-c634a54898a6-tls-certs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:00.451935 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.451932    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-model-cache\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:00.451935 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.451942    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6fv5\" (UniqueName: \"kubernetes.io/projected/a3d42637-3c0c-4508-a246-c634a54898a6-kube-api-access-s6fv5\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:00.452117 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.451951    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-dshm\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:00.452117 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.451960    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d42637-3c0c-4508-a246-c634a54898a6-home\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:00.820298 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:00.820251    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb" podUID="4c711639-401d-429e-acb0-cf79bf614b11" containerName="main" containerID="cri-o://7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6" gracePeriod=45
Jul 14 16:33:01.073750 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.073696    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:33:01.098256 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.098220    2560 generic.go:356] "Generic (PLEG): container finished" podID="4c711639-401d-429e-acb0-cf79bf614b11" containerID="7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6" exitCode=0
Jul 14 16:33:01.098388 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.098263    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"
Jul 14 16:33:01.098388 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.098304    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb" event={"ID":"4c711639-401d-429e-acb0-cf79bf614b11","Type":"ContainerDied","Data":"7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6"}
Jul 14 16:33:01.098388 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.098344    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb" event={"ID":"4c711639-401d-429e-acb0-cf79bf614b11","Type":"ContainerDied","Data":"bf57073a26f822d275e2d21f083bbccdc6e6ce3a8d20317fae09b78076073668"}
Jul 14 16:33:01.098388 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.098368    2560 scope.go:117] "RemoveContainer" containerID="7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6"
Jul 14 16:33:01.100032 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.100007    2560 generic.go:356] "Generic (PLEG): container finished" podID="a3d42637-3c0c-4508-a246-c634a54898a6" containerID="da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa" exitCode=0
Jul 14 16:33:01.100131 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.100068    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm" event={"ID":"a3d42637-3c0c-4508-a246-c634a54898a6","Type":"ContainerDied","Data":"da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa"}
Jul 14 16:33:01.100131 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.100094    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm" event={"ID":"a3d42637-3c0c-4508-a246-c634a54898a6","Type":"ContainerDied","Data":"6ab3561b40707414672423c299b2a87cbe8b325a023b6c50c6599ff74bee7066"}
Jul 14 16:33:01.100211 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.100156    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"
Jul 14 16:33:01.108279 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.108263    2560 scope.go:117] "RemoveContainer" containerID="7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6"
Jul 14 16:33:01.108582 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:33:01.108565    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6\": container with ID starting with 7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6 not found: ID does not exist" containerID="7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6"
Jul 14 16:33:01.108657 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.108592    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6"} err="failed to get container status \"7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6\": rpc error: code = NotFound desc = could not find container \"7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6\": container with ID starting with 7790b97dc6217acd79740b72af20a0ce0bd4dec8f03c2021ad230531814b38d6 not found: ID does not exist"
Jul 14 16:33:01.108657 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.108615    2560 scope.go:117] "RemoveContainer" containerID="da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa"
Jul 14 16:33:01.117401 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.117377    2560 scope.go:117] "RemoveContainer" containerID="da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa"
Jul 14 16:33:01.117696 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:33:01.117669    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa\": container with ID starting with da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa not found: ID does not exist" containerID="da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa"
Jul 14 16:33:01.117792 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.117704    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa"} err="failed to get container status \"da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa\": rpc error: code = NotFound desc = could not find container \"da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa\": container with ID starting with da19cec36f5b8462900b06c180f3cd8542ef9897a6538130b96e0b58c3080afa not found: ID does not exist"
Jul 14 16:33:01.119052 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.119035    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"]
Jul 14 16:33:01.122604 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.122585    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/auth-test-model-21b35e-kserve-6b684dc74c-kkkhm"]
Jul 14 16:33:01.159481 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.159434    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6g4d\" (UniqueName: \"kubernetes.io/projected/4c711639-401d-429e-acb0-cf79bf614b11-kube-api-access-q6g4d\") pod \"4c711639-401d-429e-acb0-cf79bf614b11\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") "
Jul 14 16:33:01.159579 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.159520    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c711639-401d-429e-acb0-cf79bf614b11-tls-certs\") pod \"4c711639-401d-429e-acb0-cf79bf614b11\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") "
Jul 14 16:33:01.159623 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.159593    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-tmp-dir\") pod \"4c711639-401d-429e-acb0-cf79bf614b11\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") "
Jul 14 16:33:01.159623 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.159619    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-dshm\") pod \"4c711639-401d-429e-acb0-cf79bf614b11\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") "
Jul 14 16:33:01.159726 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.159646    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-home\") pod \"4c711639-401d-429e-acb0-cf79bf614b11\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") "
Jul 14 16:33:01.159726 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.159660    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-model-cache\") pod \"4c711639-401d-429e-acb0-cf79bf614b11\" (UID: \"4c711639-401d-429e-acb0-cf79bf614b11\") "
Jul 14 16:33:01.159914 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.159887    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c711639-401d-429e-acb0-cf79bf614b11" (UID: "4c711639-401d-429e-acb0-cf79bf614b11"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:33:01.160010 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.159959    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-home" (OuterVolumeSpecName: "home") pod "4c711639-401d-429e-acb0-cf79bf614b11" (UID: "4c711639-401d-429e-acb0-cf79bf614b11"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:33:01.160109 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.160044    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c711639-401d-429e-acb0-cf79bf614b11" (UID: "4c711639-401d-429e-acb0-cf79bf614b11"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:33:01.161857 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.161838    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-dshm" (OuterVolumeSpecName: "dshm") pod "4c711639-401d-429e-acb0-cf79bf614b11" (UID: "4c711639-401d-429e-acb0-cf79bf614b11"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:33:01.161857 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.161843    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c711639-401d-429e-acb0-cf79bf614b11-kube-api-access-q6g4d" (OuterVolumeSpecName: "kube-api-access-q6g4d") pod "4c711639-401d-429e-acb0-cf79bf614b11" (UID: "4c711639-401d-429e-acb0-cf79bf614b11"). InnerVolumeSpecName "kube-api-access-q6g4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:33:01.161956 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.161851    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c711639-401d-429e-acb0-cf79bf614b11-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c711639-401d-429e-acb0-cf79bf614b11" (UID: "4c711639-401d-429e-acb0-cf79bf614b11"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:33:01.260889 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.260854    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-tmp-dir\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:01.260889 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.260883    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-dshm\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:01.260889 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.260891    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-home\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:01.261109 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.260899    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c711639-401d-429e-acb0-cf79bf614b11-model-cache\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:01.261109 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.260908    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6g4d\" (UniqueName: \"kubernetes.io/projected/4c711639-401d-429e-acb0-cf79bf614b11-kube-api-access-q6g4d\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:01.261109 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.260916    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c711639-401d-429e-acb0-cf79bf614b11-tls-certs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:33:01.420919 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.420893    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"]
Jul 14 16:33:01.422980 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:01.422960    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/auth-test-model-4e27c0-kserve-5456b6f9ff-fm5rb"]
Jul 14 16:33:03.032766 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:03.032731    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c711639-401d-429e-acb0-cf79bf614b11" path="/var/lib/kubelet/pods/4c711639-401d-429e-acb0-cf79bf614b11/volumes"
Jul 14 16:33:03.033133 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:03.033114    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3d42637-3c0c-4508-a246-c634a54898a6" path="/var/lib/kubelet/pods/a3d42637-3c0c-4508-a246-c634a54898a6/volumes"
Jul 14 16:33:28.503349 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.503260    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"]
Jul 14 16:33:28.503947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.503682    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerName="cleanup"
Jul 14 16:33:28.503947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.503694    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerName="cleanup"
Jul 14 16:33:28.503947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.503711    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c711639-401d-429e-acb0-cf79bf614b11" containerName="main"
Jul 14 16:33:28.503947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.503717    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c711639-401d-429e-acb0-cf79bf614b11" containerName="main"
Jul 14 16:33:28.503947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.503732    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3d42637-3c0c-4508-a246-c634a54898a6" containerName="main"
Jul 14 16:33:28.503947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.503738    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3d42637-3c0c-4508-a246-c634a54898a6" containerName="main"
Jul 14 16:33:28.503947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.503795    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c711639-401d-429e-acb0-cf79bf614b11" containerName="main"
Jul 14 16:33:28.503947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.503804    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="37ea790f-efbe-4a06-8130-0a5d98dd7fc1" containerName="cleanup"
Jul 14 16:33:28.503947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.503811    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3d42637-3c0c-4508-a246-c634a54898a6" containerName="main"
Jul 14 16:33:28.507204 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.507183    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.511125 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.511104    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"openshift-service-ca.crt\""
Jul 14 16:33:28.511274 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.511109    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"default-dockercfg-78952\""
Jul 14 16:33:28.511274 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.511109    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"sub-test-model-4e27c0-kserve-self-signed-certs\""
Jul 14 16:33:28.511274 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.511150    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"kube-root-ca.crt\""
Jul 14 16:33:28.516549 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.516514    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"]
Jul 14 16:33:28.618323 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.618295    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-model-cache\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.618515 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.618333    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-tmp-dir\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.618515 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.618357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2czfs\" (UniqueName: \"kubernetes.io/projected/e3db6d10-a005-4b32-8a8f-df192a061908-kube-api-access-2czfs\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.618515 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.618398    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-home\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.618515 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.618418    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3db6d10-a005-4b32-8a8f-df192a061908-tls-certs\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.618696 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.618524    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-dshm\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.719116 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.719086    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-tmp-dir\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.719280 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.719126    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2czfs\" (UniqueName: \"kubernetes.io/projected/e3db6d10-a005-4b32-8a8f-df192a061908-kube-api-access-2czfs\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.719280 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.719154    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-home\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.719280 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.719180    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3db6d10-a005-4b32-8a8f-df192a061908-tls-certs\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.719280 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.719230    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-dshm\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.719530 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.719436    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-model-cache\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.719657 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.719636    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-home\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.719724 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.719665    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-tmp-dir\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.719724 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.719716    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-model-cache\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.721576 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.721560    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-dshm\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.721924 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.721906    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3db6d10-a005-4b32-8a8f-df192a061908-tls-certs\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.731426 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.731399    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2czfs\" (UniqueName: \"kubernetes.io/projected/e3db6d10-a005-4b32-8a8f-df192a061908-kube-api-access-2czfs\") pod \"sub-test-model-4e27c0-kserve-d4c986c85-sdhxg\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") " pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.817646 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.817574    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:28.953140 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:28.953110    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"]
Jul 14 16:33:28.954051 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:33:28.954030    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3db6d10_a005_4b32_8a8f_df192a061908.slice/crio-631148ec2f6291fb44824624f632e8a5ef70ae8b3c2ed4d26ea0a235d5d635ae WatchSource:0}: Error finding container 631148ec2f6291fb44824624f632e8a5ef70ae8b3c2ed4d26ea0a235d5d635ae: Status 404 returned error can't find the container with id 631148ec2f6291fb44824624f632e8a5ef70ae8b3c2ed4d26ea0a235d5d635ae
Jul 14 16:33:29.257433 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:29.257393    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg" event={"ID":"e3db6d10-a005-4b32-8a8f-df192a061908","Type":"ContainerStarted","Data":"066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd"}
Jul 14 16:33:29.257640 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:29.257463    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg" event={"ID":"e3db6d10-a005-4b32-8a8f-df192a061908","Type":"ContainerStarted","Data":"631148ec2f6291fb44824624f632e8a5ef70ae8b3c2ed4d26ea0a235d5d635ae"}
Jul 14 16:33:38.818432 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:38.818396    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:38.818432 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:38.818439    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:38.831062 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:38.831035    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:33:38.850523 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:38.850473    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg" podStartSLOduration=10.850436853 podStartE2EDuration="10.850436853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:33:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:33:29.277011108 +0000 UTC m=+2792.796378547" watchObservedRunningTime="2026-07-14 16:33:38.850436853 +0000 UTC m=+2802.369804274"
Jul 14 16:33:39.324933 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:33:39.324905    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:34:09.019125 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:09.019091    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"]
Jul 14 16:34:24.039481 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.039422    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg" podUID="e3db6d10-a005-4b32-8a8f-df192a061908" containerName="main" containerID="cri-o://066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd" gracePeriod=45
Jul 14 16:34:24.292138 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.292074    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:34:24.355132 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355106    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-model-cache\") pod \"e3db6d10-a005-4b32-8a8f-df192a061908\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") "
Jul 14 16:34:24.355296 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355143    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3db6d10-a005-4b32-8a8f-df192a061908-tls-certs\") pod \"e3db6d10-a005-4b32-8a8f-df192a061908\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") "
Jul 14 16:34:24.355296 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355164    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-tmp-dir\") pod \"e3db6d10-a005-4b32-8a8f-df192a061908\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") "
Jul 14 16:34:24.355296 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355228    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-home\") pod \"e3db6d10-a005-4b32-8a8f-df192a061908\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") "
Jul 14 16:34:24.355418 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355312    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-dshm\") pod \"e3db6d10-a005-4b32-8a8f-df192a061908\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") "
Jul 14 16:34:24.355418 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355349    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2czfs\" (UniqueName: \"kubernetes.io/projected/e3db6d10-a005-4b32-8a8f-df192a061908-kube-api-access-2czfs\") pod \"e3db6d10-a005-4b32-8a8f-df192a061908\" (UID: \"e3db6d10-a005-4b32-8a8f-df192a061908\") "
Jul 14 16:34:24.355530 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355418    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-model-cache" (OuterVolumeSpecName: "model-cache") pod "e3db6d10-a005-4b32-8a8f-df192a061908" (UID: "e3db6d10-a005-4b32-8a8f-df192a061908"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:34:24.355530 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355483    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e3db6d10-a005-4b32-8a8f-df192a061908" (UID: "e3db6d10-a005-4b32-8a8f-df192a061908"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:34:24.355650 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355535    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-home" (OuterVolumeSpecName: "home") pod "e3db6d10-a005-4b32-8a8f-df192a061908" (UID: "e3db6d10-a005-4b32-8a8f-df192a061908"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:34:24.355794 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355772    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-home\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:34:24.355794 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355792    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-model-cache\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:34:24.356228 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.355805    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-tmp-dir\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:34:24.357245 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.357222    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3db6d10-a005-4b32-8a8f-df192a061908-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e3db6d10-a005-4b32-8a8f-df192a061908" (UID: "e3db6d10-a005-4b32-8a8f-df192a061908"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:34:24.357355 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.357342    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-dshm" (OuterVolumeSpecName: "dshm") pod "e3db6d10-a005-4b32-8a8f-df192a061908" (UID: "e3db6d10-a005-4b32-8a8f-df192a061908"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:34:24.357435 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.357415    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3db6d10-a005-4b32-8a8f-df192a061908-kube-api-access-2czfs" (OuterVolumeSpecName: "kube-api-access-2czfs") pod "e3db6d10-a005-4b32-8a8f-df192a061908" (UID: "e3db6d10-a005-4b32-8a8f-df192a061908"). InnerVolumeSpecName "kube-api-access-2czfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:34:24.456646 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.456618    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3db6d10-a005-4b32-8a8f-df192a061908-dshm\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:34:24.456646 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.456643    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2czfs\" (UniqueName: \"kubernetes.io/projected/e3db6d10-a005-4b32-8a8f-df192a061908-kube-api-access-2czfs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:34:24.456814 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.456653    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3db6d10-a005-4b32-8a8f-df192a061908-tls-certs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:34:24.559758 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.559677    2560 generic.go:356] "Generic (PLEG): container finished" podID="e3db6d10-a005-4b32-8a8f-df192a061908" containerID="066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd" exitCode=0
Jul 14 16:34:24.559758 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.559715    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg" event={"ID":"e3db6d10-a005-4b32-8a8f-df192a061908","Type":"ContainerDied","Data":"066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd"}
Jul 14 16:34:24.559758 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.559719    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"
Jul 14 16:34:24.559758 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.559742    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg" event={"ID":"e3db6d10-a005-4b32-8a8f-df192a061908","Type":"ContainerDied","Data":"631148ec2f6291fb44824624f632e8a5ef70ae8b3c2ed4d26ea0a235d5d635ae"}
Jul 14 16:34:24.560026 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.559764    2560 scope.go:117] "RemoveContainer" containerID="066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd"
Jul 14 16:34:24.574924 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.570179    2560 scope.go:117] "RemoveContainer" containerID="066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd"
Jul 14 16:34:24.574924 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:34:24.570587    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd\": container with ID starting with 066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd not found: ID does not exist" containerID="066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd"
Jul 14 16:34:24.574924 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.570626    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd"} err="failed to get container status \"066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd\": rpc error: code = NotFound desc = could not find container \"066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd\": container with ID starting with 066fdcce1e5fe669424e6d5b9ad5da2b16c4f925b89df6bff610bc16f138e9bd not found: ID does not exist"
Jul 14 16:34:24.582568 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.582544    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"]
Jul 14 16:34:24.588413 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:24.588390    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/sub-test-model-4e27c0-kserve-d4c986c85-sdhxg"]
Jul 14 16:34:25.032130 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:25.032092    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3db6d10-a005-4b32-8a8f-df192a061908" path="/var/lib/kubelet/pods/e3db6d10-a005-4b32-8a8f-df192a061908/volumes"
Jul 14 16:34:26.502081 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.502044    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"]
Jul 14 16:34:26.502719 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.502702    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3db6d10-a005-4b32-8a8f-df192a061908" containerName="main"
Jul 14 16:34:26.502773 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.502723    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3db6d10-a005-4b32-8a8f-df192a061908" containerName="main"
Jul 14 16:34:26.502845 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.502834    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3db6d10-a005-4b32-8a8f-df192a061908" containerName="main"
Jul 14 16:34:26.507589 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.507567    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.511409 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.511385    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"openshift-service-ca.crt\""
Jul 14 16:34:26.511828 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.511405    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"rate-test-model-21b35e-kserve-self-signed-certs\""
Jul 14 16:34:26.511828 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.511509    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"default-dockercfg-sf92j\""
Jul 14 16:34:26.511828 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.511534    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-21b35e\"/\"kube-root-ca.crt\""
Jul 14 16:34:26.514266 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.514234    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"]
Jul 14 16:34:26.574022 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.573990    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-home\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.574205 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.574057    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-model-cache\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.574205 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.574104    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab79a609-82a0-4c60-976b-68a08c4ab12f-tls-certs\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.574205 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.574134    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-tmp-dir\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.574205 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.574165    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-dshm\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.574205 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.574183    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqvjh\" (UniqueName: \"kubernetes.io/projected/ab79a609-82a0-4c60-976b-68a08c4ab12f-kube-api-access-qqvjh\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.675152 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.675124    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-home\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.675323 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.675174    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-model-cache\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.675323 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.675293    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab79a609-82a0-4c60-976b-68a08c4ab12f-tls-certs\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.675428 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.675326    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-tmp-dir\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.675428 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.675360    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-dshm\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.675428 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.675378    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqvjh\" (UniqueName: \"kubernetes.io/projected/ab79a609-82a0-4c60-976b-68a08c4ab12f-kube-api-access-qqvjh\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.675634 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.675579    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-model-cache\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.675634 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.675607    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-home\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.675752 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.675682    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-tmp-dir\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.677887 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.677865    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-dshm\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.678009 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.677905    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab79a609-82a0-4c60-976b-68a08c4ab12f-tls-certs\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.686632 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.686611    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqvjh\" (UniqueName: \"kubernetes.io/projected/ab79a609-82a0-4c60-976b-68a08c4ab12f-kube-api-access-qqvjh\") pod \"rate-test-model-21b35e-kserve-86bb9df44c-cbfhb\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") " pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.819065 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.818987    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:26.955565 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:26.955534    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"]
Jul 14 16:34:26.956717 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:34:26.956688    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab79a609_82a0_4c60_976b_68a08c4ab12f.slice/crio-c6f2700441d8191781f337e9e2793a013260e23584501ef294a813351e72e10c WatchSource:0}: Error finding container c6f2700441d8191781f337e9e2793a013260e23584501ef294a813351e72e10c: Status 404 returned error can't find the container with id c6f2700441d8191781f337e9e2793a013260e23584501ef294a813351e72e10c
Jul 14 16:34:27.579664 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:27.579633    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb" event={"ID":"ab79a609-82a0-4c60-976b-68a08c4ab12f","Type":"ContainerStarted","Data":"95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6"}
Jul 14 16:34:27.580090 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:27.579675    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb" event={"ID":"ab79a609-82a0-4c60-976b-68a08c4ab12f","Type":"ContainerStarted","Data":"c6f2700441d8191781f337e9e2793a013260e23584501ef294a813351e72e10c"}
Jul 14 16:34:27.599568 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:27.599516    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb" podStartSLOduration=1.599495557 podStartE2EDuration="1.599495557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:34:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:34:27.598568025 +0000 UTC m=+2851.117935447" watchObservedRunningTime="2026-07-14 16:34:27.599495557 +0000 UTC m=+2851.118862980"
Jul 14 16:34:36.820011 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:36.819970    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:36.820506 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:36.820029    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:36.832959 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:36.832928    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:34:37.644706 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:34:37.644677    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:35:01.910914 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:01.910879    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"]
Jul 14 16:35:01.915681 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:01.915663    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:01.921418 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:01.921392    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"kube-root-ca.crt\""
Jul 14 16:35:01.921418 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:01.921406    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"rate-test-model-4e27c0-kserve-self-signed-certs\""
Jul 14 16:35:01.921627 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:01.921417    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"openshift-service-ca.crt\""
Jul 14 16:35:01.921627 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:01.921483    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4e27c0\"/\"default-dockercfg-78952\""
Jul 14 16:35:01.931929 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:01.931903    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"]
Jul 14 16:35:02.013879 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.013843    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49426232-2389-489e-b544-3c76f662efdf-tls-certs\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.014047 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.013884    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-model-cache\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.014047 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.013913    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-tmp-dir\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.014047 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.013993    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-dshm\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.014158 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.014090    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-home\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.014158 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.014119    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdlx4\" (UniqueName: \"kubernetes.io/projected/49426232-2389-489e-b544-3c76f662efdf-kube-api-access-qdlx4\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.114966 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.114931    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-home\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.115164 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.114983    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdlx4\" (UniqueName: \"kubernetes.io/projected/49426232-2389-489e-b544-3c76f662efdf-kube-api-access-qdlx4\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.115164 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.115002    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49426232-2389-489e-b544-3c76f662efdf-tls-certs\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.115164 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.115024    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-model-cache\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.115378 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.115166    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-tmp-dir\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.115378 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.115258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-dshm\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.115532 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.115400    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-home\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.115532 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.115431    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-model-cache\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.115532 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.115525    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-tmp-dir\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.117500 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.117478    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-dshm\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.117678 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.117663    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49426232-2389-489e-b544-3c76f662efdf-tls-certs\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.125522 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.125501    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdlx4\" (UniqueName: \"kubernetes.io/projected/49426232-2389-489e-b544-3c76f662efdf-kube-api-access-qdlx4\") pod \"rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") " pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.225494 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.225397    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:02.364432 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.364407    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"]
Jul 14 16:35:02.365993 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:35:02.365969    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49426232_2389_489e_b544_3c76f662efdf.slice/crio-a1994227c166c4d12f74429748a92f93b7a9422291680bbe8b1c7ca9c9ac4ea3 WatchSource:0}: Error finding container a1994227c166c4d12f74429748a92f93b7a9422291680bbe8b1c7ca9c9ac4ea3: Status 404 returned error can't find the container with id a1994227c166c4d12f74429748a92f93b7a9422291680bbe8b1c7ca9c9ac4ea3
Jul 14 16:35:02.778184 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.778152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729" event={"ID":"49426232-2389-489e-b544-3c76f662efdf","Type":"ContainerStarted","Data":"e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5"}
Jul 14 16:35:02.778360 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.778193    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729" event={"ID":"49426232-2389-489e-b544-3c76f662efdf","Type":"ContainerStarted","Data":"a1994227c166c4d12f74429748a92f93b7a9422291680bbe8b1c7ca9c9ac4ea3"}
Jul 14 16:35:02.798218 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:02.798170    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729" podStartSLOduration=1.7981543439999998 podStartE2EDuration="1.798154344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 16:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:35:02.797554667 +0000 UTC m=+2886.316922086" watchObservedRunningTime="2026-07-14 16:35:02.798154344 +0000 UTC m=+2886.317521764"
Jul 14 16:35:12.226103 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:12.226066    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:12.226590 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:12.226123    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:12.238317 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:12.238295    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:12.842781 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:12.842752    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:35:49.470836 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:49.470802    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"]
Jul 14 16:35:50.476737 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:35:50.476704    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"]
Jul 14 16:36:04.491254 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.491180    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb" podUID="ab79a609-82a0-4c60-976b-68a08c4ab12f" containerName="main" containerID="cri-o://95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6" gracePeriod=45
Jul 14 16:36:04.740720 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.740695    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:36:04.805058 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.804984    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-home\") pod \"ab79a609-82a0-4c60-976b-68a08c4ab12f\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") "
Jul 14 16:36:04.805204 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.805074    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-model-cache\") pod \"ab79a609-82a0-4c60-976b-68a08c4ab12f\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") "
Jul 14 16:36:04.805204 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.805108    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqvjh\" (UniqueName: \"kubernetes.io/projected/ab79a609-82a0-4c60-976b-68a08c4ab12f-kube-api-access-qqvjh\") pod \"ab79a609-82a0-4c60-976b-68a08c4ab12f\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") "
Jul 14 16:36:04.805204 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.805139    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-dshm\") pod \"ab79a609-82a0-4c60-976b-68a08c4ab12f\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") "
Jul 14 16:36:04.805204 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.805160    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab79a609-82a0-4c60-976b-68a08c4ab12f-tls-certs\") pod \"ab79a609-82a0-4c60-976b-68a08c4ab12f\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") "
Jul 14 16:36:04.805204 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.805188    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-tmp-dir\") pod \"ab79a609-82a0-4c60-976b-68a08c4ab12f\" (UID: \"ab79a609-82a0-4c60-976b-68a08c4ab12f\") "
Jul 14 16:36:04.805598 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.805289    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-home" (OuterVolumeSpecName: "home") pod "ab79a609-82a0-4c60-976b-68a08c4ab12f" (UID: "ab79a609-82a0-4c60-976b-68a08c4ab12f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:36:04.805598 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.805389    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-model-cache" (OuterVolumeSpecName: "model-cache") pod "ab79a609-82a0-4c60-976b-68a08c4ab12f" (UID: "ab79a609-82a0-4c60-976b-68a08c4ab12f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:36:04.805598 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.805547    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ab79a609-82a0-4c60-976b-68a08c4ab12f" (UID: "ab79a609-82a0-4c60-976b-68a08c4ab12f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:36:04.805750 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.805701    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-tmp-dir\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:04.805750 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.805719    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-home\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:04.805750 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.805732    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-model-cache\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:04.807377 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.807355    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-dshm" (OuterVolumeSpecName: "dshm") pod "ab79a609-82a0-4c60-976b-68a08c4ab12f" (UID: "ab79a609-82a0-4c60-976b-68a08c4ab12f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:36:04.807479 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.807438    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab79a609-82a0-4c60-976b-68a08c4ab12f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ab79a609-82a0-4c60-976b-68a08c4ab12f" (UID: "ab79a609-82a0-4c60-976b-68a08c4ab12f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:36:04.807524 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.807485    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab79a609-82a0-4c60-976b-68a08c4ab12f-kube-api-access-qqvjh" (OuterVolumeSpecName: "kube-api-access-qqvjh") pod "ab79a609-82a0-4c60-976b-68a08c4ab12f" (UID: "ab79a609-82a0-4c60-976b-68a08c4ab12f"). InnerVolumeSpecName "kube-api-access-qqvjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:36:04.906380 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.906347    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqvjh\" (UniqueName: \"kubernetes.io/projected/ab79a609-82a0-4c60-976b-68a08c4ab12f-kube-api-access-qqvjh\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:04.906380 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.906374    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab79a609-82a0-4c60-976b-68a08c4ab12f-dshm\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:04.906587 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:04.906387    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab79a609-82a0-4c60-976b-68a08c4ab12f-tls-certs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:05.113559 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.113526    2560 generic.go:356] "Generic (PLEG): container finished" podID="ab79a609-82a0-4c60-976b-68a08c4ab12f" containerID="95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6" exitCode=0
Jul 14 16:36:05.113719 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.113567    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb" event={"ID":"ab79a609-82a0-4c60-976b-68a08c4ab12f","Type":"ContainerDied","Data":"95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6"}
Jul 14 16:36:05.113719 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.113568    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"
Jul 14 16:36:05.113719 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.113595    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb" event={"ID":"ab79a609-82a0-4c60-976b-68a08c4ab12f","Type":"ContainerDied","Data":"c6f2700441d8191781f337e9e2793a013260e23584501ef294a813351e72e10c"}
Jul 14 16:36:05.113719 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.113621    2560 scope.go:117] "RemoveContainer" containerID="95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6"
Jul 14 16:36:05.122667 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.122643    2560 scope.go:117] "RemoveContainer" containerID="95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6"
Jul 14 16:36:05.122959 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:36:05.122941    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6\": container with ID starting with 95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6 not found: ID does not exist" containerID="95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6"
Jul 14 16:36:05.123044 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.122970    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6"} err="failed to get container status \"95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6\": rpc error: code = NotFound desc = could not find container \"95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6\": container with ID starting with 95ea9d034c055ee505593c18c080b5fc78cf8cdb402e28e7ca7eb3de79d2c1a6 not found: ID does not exist"
Jul 14 16:36:05.132686 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.132665    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"]
Jul 14 16:36:05.140063 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.140042    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-21b35e/rate-test-model-21b35e-kserve-86bb9df44c-cbfhb"]
Jul 14 16:36:05.496917 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.496811    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729" podUID="49426232-2389-489e-b544-3c76f662efdf" containerName="main" containerID="cri-o://e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5" gracePeriod=45
Jul 14 16:36:05.757467 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.757386    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:36:05.813614 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.813584    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdlx4\" (UniqueName: \"kubernetes.io/projected/49426232-2389-489e-b544-3c76f662efdf-kube-api-access-qdlx4\") pod \"49426232-2389-489e-b544-3c76f662efdf\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") "
Jul 14 16:36:05.813814 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.813671    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-tmp-dir\") pod \"49426232-2389-489e-b544-3c76f662efdf\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") "
Jul 14 16:36:05.813814 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.813773    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-home\") pod \"49426232-2389-489e-b544-3c76f662efdf\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") "
Jul 14 16:36:05.813814 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.813811    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-dshm\") pod \"49426232-2389-489e-b544-3c76f662efdf\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") "
Jul 14 16:36:05.813988 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.813831    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-model-cache\") pod \"49426232-2389-489e-b544-3c76f662efdf\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") "
Jul 14 16:36:05.813988 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.813898    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "49426232-2389-489e-b544-3c76f662efdf" (UID: "49426232-2389-489e-b544-3c76f662efdf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:36:05.813988 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.813891    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49426232-2389-489e-b544-3c76f662efdf-tls-certs\") pod \"49426232-2389-489e-b544-3c76f662efdf\" (UID: \"49426232-2389-489e-b544-3c76f662efdf\") "
Jul 14 16:36:05.813988 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.813971    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-home" (OuterVolumeSpecName: "home") pod "49426232-2389-489e-b544-3c76f662efdf" (UID: "49426232-2389-489e-b544-3c76f662efdf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:36:05.814177 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.814156    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-model-cache" (OuterVolumeSpecName: "model-cache") pod "49426232-2389-489e-b544-3c76f662efdf" (UID: "49426232-2389-489e-b544-3c76f662efdf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:36:05.814298 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.814284    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-tmp-dir\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:05.814356 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.814300    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-home\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:05.814356 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.814309    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-model-cache\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:05.815976 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.815955    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49426232-2389-489e-b544-3c76f662efdf-kube-api-access-qdlx4" (OuterVolumeSpecName: "kube-api-access-qdlx4") pod "49426232-2389-489e-b544-3c76f662efdf" (UID: "49426232-2389-489e-b544-3c76f662efdf"). InnerVolumeSpecName "kube-api-access-qdlx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:36:05.816087 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.816034    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49426232-2389-489e-b544-3c76f662efdf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "49426232-2389-489e-b544-3c76f662efdf" (UID: "49426232-2389-489e-b544-3c76f662efdf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:36:05.816087 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.816054    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-dshm" (OuterVolumeSpecName: "dshm") pod "49426232-2389-489e-b544-3c76f662efdf" (UID: "49426232-2389-489e-b544-3c76f662efdf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 16:36:05.914942 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.914907    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49426232-2389-489e-b544-3c76f662efdf-dshm\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:05.914942 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.914933    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49426232-2389-489e-b544-3c76f662efdf-tls-certs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:05.914942 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:05.914945    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdlx4\" (UniqueName: \"kubernetes.io/projected/49426232-2389-489e-b544-3c76f662efdf-kube-api-access-qdlx4\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:06.120478 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:06.120428    2560 generic.go:356] "Generic (PLEG): container finished" podID="49426232-2389-489e-b544-3c76f662efdf" containerID="e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5" exitCode=0
Jul 14 16:36:06.120656 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:06.120490    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"
Jul 14 16:36:06.120656 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:06.120517    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729" event={"ID":"49426232-2389-489e-b544-3c76f662efdf","Type":"ContainerDied","Data":"e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5"}
Jul 14 16:36:06.120656 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:06.120565    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729" event={"ID":"49426232-2389-489e-b544-3c76f662efdf","Type":"ContainerDied","Data":"a1994227c166c4d12f74429748a92f93b7a9422291680bbe8b1c7ca9c9ac4ea3"}
Jul 14 16:36:06.120656 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:06.120591    2560 scope.go:117] "RemoveContainer" containerID="e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5"
Jul 14 16:36:06.130642 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:06.130624    2560 scope.go:117] "RemoveContainer" containerID="e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5"
Jul 14 16:36:06.130899 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:36:06.130882    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5\": container with ID starting with e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5 not found: ID does not exist" containerID="e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5"
Jul 14 16:36:06.130953 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:06.130907    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5"} err="failed to get container status \"e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5\": rpc error: code = NotFound desc = could not find container \"e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5\": container with ID starting with e604aacbf888b533d5835d545d1f8d9d1be7330e920ac34273a11e04090d4de5 not found: ID does not exist"
Jul 14 16:36:06.147970 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:06.147948    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"]
Jul 14 16:36:06.151198 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:06.151178    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4e27c0/rate-test-model-4e27c0-kserve-7dd5dd8bf5-6d729"]
Jul 14 16:36:07.033357 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:07.033325    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49426232-2389-489e-b544-3c76f662efdf" path="/var/lib/kubelet/pods/49426232-2389-489e-b544-3c76f662efdf/volumes"
Jul 14 16:36:07.033747 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:07.033730    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab79a609-82a0-4c60-976b-68a08c4ab12f" path="/var/lib/kubelet/pods/ab79a609-82a0-4c60-976b-68a08c4ab12f/volumes"
Jul 14 16:36:15.459289 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.459253    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7422-139946477613392"]
Jul 14 16:36:15.459726 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.459712    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49426232-2389-489e-b544-3c76f662efdf" containerName="main"
Jul 14 16:36:15.459726 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.459728    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="49426232-2389-489e-b544-3c76f662efdf" containerName="main"
Jul 14 16:36:15.459802 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.459748    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab79a609-82a0-4c60-976b-68a08c4ab12f" containerName="main"
Jul 14 16:36:15.459802 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.459755    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab79a609-82a0-4c60-976b-68a08c4ab12f" containerName="main"
Jul 14 16:36:15.459864 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.459826    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab79a609-82a0-4c60-976b-68a08c4ab12f" containerName="main"
Jul 14 16:36:15.459864 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.459837    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="49426232-2389-489e-b544-3c76f662efdf" containerName="main"
Jul 14 16:36:15.464246 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.464223    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477613392"
Jul 14 16:36:15.466855 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.466830    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-bzk99\""
Jul 14 16:36:15.469568 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.469545    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7422-139946477613392"]
Jul 14 16:36:15.503584 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.503550    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx2bk\" (UniqueName: \"kubernetes.io/projected/df02682b-0a6b-4ddb-aa5f-8d7f86d0913f-kube-api-access-wx2bk\") pod \"test-curl-7422-139946477613392\" (UID: \"df02682b-0a6b-4ddb-aa5f-8d7f86d0913f\") " pod="openshift-ingress/test-curl-7422-139946477613392"
Jul 14 16:36:15.604016 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.603981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx2bk\" (UniqueName: \"kubernetes.io/projected/df02682b-0a6b-4ddb-aa5f-8d7f86d0913f-kube-api-access-wx2bk\") pod \"test-curl-7422-139946477613392\" (UID: \"df02682b-0a6b-4ddb-aa5f-8d7f86d0913f\") " pod="openshift-ingress/test-curl-7422-139946477613392"
Jul 14 16:36:15.612717 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.612695    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx2bk\" (UniqueName: \"kubernetes.io/projected/df02682b-0a6b-4ddb-aa5f-8d7f86d0913f-kube-api-access-wx2bk\") pod \"test-curl-7422-139946477613392\" (UID: \"df02682b-0a6b-4ddb-aa5f-8d7f86d0913f\") " pod="openshift-ingress/test-curl-7422-139946477613392"
Jul 14 16:36:15.774529 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.774432    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477613392"
Jul 14 16:36:15.910552 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.910521    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7422-139946477613392"]
Jul 14 16:36:15.911845 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:36:15.911815    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf02682b_0a6b_4ddb_aa5f_8d7f86d0913f.slice/crio-b3f5faa83873160e2ecbb9a571888c670b88acf607e4a1256d03b625857269b8 WatchSource:0}: Error finding container b3f5faa83873160e2ecbb9a571888c670b88acf607e4a1256d03b625857269b8: Status 404 returned error can't find the container with id b3f5faa83873160e2ecbb9a571888c670b88acf607e4a1256d03b625857269b8
Jul 14 16:36:15.913099 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:15.913081    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 16:36:16.181066 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:16.181029    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7422-139946477613392" event={"ID":"df02682b-0a6b-4ddb-aa5f-8d7f86d0913f","Type":"ContainerStarted","Data":"b3f5faa83873160e2ecbb9a571888c670b88acf607e4a1256d03b625857269b8"}
Jul 14 16:36:17.189421 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:17.189385    2560 generic.go:356] "Generic (PLEG): container finished" podID="df02682b-0a6b-4ddb-aa5f-8d7f86d0913f" containerID="ed501d3ab7ec7816e759c5a5cd7d5c2613388d978c6af924bf903d9512869c76" exitCode=0
Jul 14 16:36:17.189421 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:17.189424    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7422-139946477613392" event={"ID":"df02682b-0a6b-4ddb-aa5f-8d7f86d0913f","Type":"ContainerDied","Data":"ed501d3ab7ec7816e759c5a5cd7d5c2613388d978c6af924bf903d9512869c76"}
Jul 14 16:36:18.346307 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.346280    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477613392"
Jul 14 16:36:18.429138 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.429108    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx2bk\" (UniqueName: \"kubernetes.io/projected/df02682b-0a6b-4ddb-aa5f-8d7f86d0913f-kube-api-access-wx2bk\") pod \"df02682b-0a6b-4ddb-aa5f-8d7f86d0913f\" (UID: \"df02682b-0a6b-4ddb-aa5f-8d7f86d0913f\") "
Jul 14 16:36:18.431351 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.431327    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df02682b-0a6b-4ddb-aa5f-8d7f86d0913f-kube-api-access-wx2bk" (OuterVolumeSpecName: "kube-api-access-wx2bk") pod "df02682b-0a6b-4ddb-aa5f-8d7f86d0913f" (UID: "df02682b-0a6b-4ddb-aa5f-8d7f86d0913f"). InnerVolumeSpecName "kube-api-access-wx2bk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:36:18.530734 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.530670    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx2bk\" (UniqueName: \"kubernetes.io/projected/df02682b-0a6b-4ddb-aa5f-8d7f86d0913f-kube-api-access-wx2bk\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:18.536189 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.536169    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7422-139946477613392_df02682b-0a6b-4ddb-aa5f-8d7f86d0913f/test-curl-7422-139946477613392/0.log"
Jul 14 16:36:18.591178 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.591149    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7422-139946477613392"]
Jul 14 16:36:18.606078 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.606060    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7422-139946477613392"]
Jul 14 16:36:18.734464 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.734419    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7422-139946477614400"]
Jul 14 16:36:18.734850 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.734837    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df02682b-0a6b-4ddb-aa5f-8d7f86d0913f" containerName="test-curl-7422-139946477613392"
Jul 14 16:36:18.734893 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.734852    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="df02682b-0a6b-4ddb-aa5f-8d7f86d0913f" containerName="test-curl-7422-139946477613392"
Jul 14 16:36:18.734943 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.734934    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="df02682b-0a6b-4ddb-aa5f-8d7f86d0913f" containerName="test-curl-7422-139946477613392"
Jul 14 16:36:18.737991 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.737976    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:18.746734 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.746710    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7422-139946477614400"]
Jul 14 16:36:18.832991 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.832914    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l8dd\" (UniqueName: \"kubernetes.io/projected/7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee-kube-api-access-6l8dd\") pod \"test-curl-7422-139946477614400\" (UID: \"7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee\") " pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:18.934338 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.934296    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l8dd\" (UniqueName: \"kubernetes.io/projected/7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee-kube-api-access-6l8dd\") pod \"test-curl-7422-139946477614400\" (UID: \"7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee\") " pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:18.943689 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:18.943660    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l8dd\" (UniqueName: \"kubernetes.io/projected/7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee-kube-api-access-6l8dd\") pod \"test-curl-7422-139946477614400\" (UID: \"7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee\") " pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:19.032305 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:19.032270    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df02682b-0a6b-4ddb-aa5f-8d7f86d0913f" path="/var/lib/kubelet/pods/df02682b-0a6b-4ddb-aa5f-8d7f86d0913f/volumes"
Jul 14 16:36:19.047518 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:19.047494    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:19.181878 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:19.181855    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7422-139946477614400"]
Jul 14 16:36:19.183032 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:36:19.183008    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b2d3fe4_6d5f_462d_9b6d_fadb9b235aee.slice/crio-a2014a1e11005f6c533aa692a7cd83110c04efbdc5b141909684cd02110fc408 WatchSource:0}: Error finding container a2014a1e11005f6c533aa692a7cd83110c04efbdc5b141909684cd02110fc408: Status 404 returned error can't find the container with id a2014a1e11005f6c533aa692a7cd83110c04efbdc5b141909684cd02110fc408
Jul 14 16:36:19.202172 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:19.202144    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477613392"
Jul 14 16:36:19.202296 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:19.202185    2560 scope.go:117] "RemoveContainer" containerID="ed501d3ab7ec7816e759c5a5cd7d5c2613388d978c6af924bf903d9512869c76"
Jul 14 16:36:19.203415 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:19.203385    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7422-139946477614400" event={"ID":"7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee","Type":"ContainerStarted","Data":"a2014a1e11005f6c533aa692a7cd83110c04efbdc5b141909684cd02110fc408"}
Jul 14 16:36:20.211993 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:20.211955    2560 generic.go:356] "Generic (PLEG): container finished" podID="7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee" containerID="9cbb02553984d43dc9c1fd25716c77c1ebae53a79f660447f84939d8dc0b6835" exitCode=0
Jul 14 16:36:20.212359 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:20.212033    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7422-139946477614400" event={"ID":"7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee","Type":"ContainerDied","Data":"9cbb02553984d43dc9c1fd25716c77c1ebae53a79f660447f84939d8dc0b6835"}
Jul 14 16:36:21.359973 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.359945    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:21.385602 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.385577    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7422-139946477614400_7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee/test-curl-7422-139946477614400/0.log"
Jul 14 16:36:21.410938 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.410912    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7422-139946477614400"]
Jul 14 16:36:21.413137 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.413117    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7422-139946477614400"]
Jul 14 16:36:21.457945 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.457923    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6l8dd\" (UniqueName: \"kubernetes.io/projected/7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee-kube-api-access-6l8dd\") pod \"7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee\" (UID: \"7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee\") "
Jul 14 16:36:21.460096 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.460071    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee-kube-api-access-6l8dd" (OuterVolumeSpecName: "kube-api-access-6l8dd") pod "7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee" (UID: "7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee"). InnerVolumeSpecName "kube-api-access-6l8dd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:36:21.544324 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.544250    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7422-139946477614400"]
Jul 14 16:36:21.544784 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.544766    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee" containerName="test-curl-7422-139946477614400"
Jul 14 16:36:21.544877 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.544786    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee" containerName="test-curl-7422-139946477614400"
Jul 14 16:36:21.544933 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.544915    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee" containerName="test-curl-7422-139946477614400"
Jul 14 16:36:21.547937 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.547918    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:21.559225 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.559200    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6l8dd\" (UniqueName: \"kubernetes.io/projected/7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee-kube-api-access-6l8dd\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:21.560399 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.560381    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7422-139946477614400"]
Jul 14 16:36:21.660367 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.660341    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmc59\" (UniqueName: \"kubernetes.io/projected/0508ceb7-da74-4cad-8359-18749ad511da-kube-api-access-nmc59\") pod \"test-curl-7422-139946477614400\" (UID: \"0508ceb7-da74-4cad-8359-18749ad511da\") " pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:21.761685 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.761661    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmc59\" (UniqueName: \"kubernetes.io/projected/0508ceb7-da74-4cad-8359-18749ad511da-kube-api-access-nmc59\") pod \"test-curl-7422-139946477614400\" (UID: \"0508ceb7-da74-4cad-8359-18749ad511da\") " pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:21.771417 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.771389    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmc59\" (UniqueName: \"kubernetes.io/projected/0508ceb7-da74-4cad-8359-18749ad511da-kube-api-access-nmc59\") pod \"test-curl-7422-139946477614400\" (UID: \"0508ceb7-da74-4cad-8359-18749ad511da\") " pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:21.856806 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.856781    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:21.989538 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:21.989509    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7422-139946477614400"]
Jul 14 16:36:21.991074 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:36:21.991045    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0508ceb7_da74_4cad_8359_18749ad511da.slice/crio-f35ec50b7ecb7acd872e2add8a8cfcc6f5d5822f28eaa69baa463b4a4f79c33b WatchSource:0}: Error finding container f35ec50b7ecb7acd872e2add8a8cfcc6f5d5822f28eaa69baa463b4a4f79c33b: Status 404 returned error can't find the container with id f35ec50b7ecb7acd872e2add8a8cfcc6f5d5822f28eaa69baa463b4a4f79c33b
Jul 14 16:36:22.231684 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:22.231601    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2014a1e11005f6c533aa692a7cd83110c04efbdc5b141909684cd02110fc408"
Jul 14 16:36:22.231684 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:22.231617    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:22.232903 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:22.232872    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7422-139946477614400" event={"ID":"0508ceb7-da74-4cad-8359-18749ad511da","Type":"ContainerStarted","Data":"f35ec50b7ecb7acd872e2add8a8cfcc6f5d5822f28eaa69baa463b4a4f79c33b"}
Jul 14 16:36:22.243968 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:22.243942    2560 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7422-139946477614400" oldPodUID="7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee" podUID="0508ceb7-da74-4cad-8359-18749ad511da"
Jul 14 16:36:23.032248 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:23.032214    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee" path="/var/lib/kubelet/pods/7b2d3fe4-6d5f-462d-9b6d-fadb9b235aee/volumes"
Jul 14 16:36:23.239778 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:23.239741    2560 generic.go:356] "Generic (PLEG): container finished" podID="0508ceb7-da74-4cad-8359-18749ad511da" containerID="8fcd0d0ab98950b2a3384bcfd64726cfa7eecc6cd0dc675d885d4727e913664e" exitCode=0
Jul 14 16:36:23.239969 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:23.239836    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7422-139946477614400" event={"ID":"0508ceb7-da74-4cad-8359-18749ad511da","Type":"ContainerDied","Data":"8fcd0d0ab98950b2a3384bcfd64726cfa7eecc6cd0dc675d885d4727e913664e"}
Jul 14 16:36:24.384091 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.384066    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:24.418493 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.418441    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7422-139946477614400_0508ceb7-da74-4cad-8359-18749ad511da/test-curl-7422-139946477614400/0.log"
Jul 14 16:36:24.462355 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.462324    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7422-139946477614400"]
Jul 14 16:36:24.467026 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.467004    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7422-139946477614400"]
Jul 14 16:36:24.487902 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.487878    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmc59\" (UniqueName: \"kubernetes.io/projected/0508ceb7-da74-4cad-8359-18749ad511da-kube-api-access-nmc59\") pod \"0508ceb7-da74-4cad-8359-18749ad511da\" (UID: \"0508ceb7-da74-4cad-8359-18749ad511da\") "
Jul 14 16:36:24.490083 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.490059    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0508ceb7-da74-4cad-8359-18749ad511da-kube-api-access-nmc59" (OuterVolumeSpecName: "kube-api-access-nmc59") pod "0508ceb7-da74-4cad-8359-18749ad511da" (UID: "0508ceb7-da74-4cad-8359-18749ad511da"). InnerVolumeSpecName "kube-api-access-nmc59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:36:24.589027 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.588996    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmc59\" (UniqueName: \"kubernetes.io/projected/0508ceb7-da74-4cad-8359-18749ad511da-kube-api-access-nmc59\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:24.698838 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.698794    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5"]
Jul 14 16:36:24.699236 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.699220    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0508ceb7-da74-4cad-8359-18749ad511da" containerName="test-curl-7422-139946477614400"
Jul 14 16:36:24.699236 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.699234    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0508ceb7-da74-4cad-8359-18749ad511da" containerName="test-curl-7422-139946477614400"
Jul 14 16:36:24.699409 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.699318    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0508ceb7-da74-4cad-8359-18749ad511da" containerName="test-curl-7422-139946477614400"
Jul 14 16:36:24.702534 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.702512    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5"
Jul 14 16:36:24.705831 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.705807    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:36:24.716523 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.716499    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5"]
Jul 14 16:36:24.790631 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.790565    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1944156a-6d7a-4466-a481-a43e9fdbc0c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5\" (UID: \"1944156a-6d7a-4466-a481-a43e9fdbc0c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5"
Jul 14 16:36:24.891733 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.891703    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1944156a-6d7a-4466-a481-a43e9fdbc0c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5\" (UID: \"1944156a-6d7a-4466-a481-a43e9fdbc0c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5"
Jul 14 16:36:24.892388 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:24.892370    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1944156a-6d7a-4466-a481-a43e9fdbc0c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5\" (UID: \"1944156a-6d7a-4466-a481-a43e9fdbc0c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5"
Jul 14 16:36:25.013803 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:25.013758    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5"
Jul 14 16:36:25.034071 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:25.034037    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0508ceb7-da74-4cad-8359-18749ad511da" path="/var/lib/kubelet/pods/0508ceb7-da74-4cad-8359-18749ad511da/volumes"
Jul 14 16:36:25.153478 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:25.153430    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5"]
Jul 14 16:36:25.155150 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:36:25.155123    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1944156a_6d7a_4466_a481_a43e9fdbc0c8.slice/crio-71a07c5279a9171d9b93e742baf7551e949cc4c4f4ddb7b56d9b4a3a600f1829 WatchSource:0}: Error finding container 71a07c5279a9171d9b93e742baf7551e949cc4c4f4ddb7b56d9b4a3a600f1829: Status 404 returned error can't find the container with id 71a07c5279a9171d9b93e742baf7551e949cc4c4f4ddb7b56d9b4a3a600f1829
Jul 14 16:36:25.252160 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:25.252129    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7422-139946477614400"
Jul 14 16:36:25.252160 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:25.252146    2560 scope.go:117] "RemoveContainer" containerID="8fcd0d0ab98950b2a3384bcfd64726cfa7eecc6cd0dc675d885d4727e913664e"
Jul 14 16:36:25.253835 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:25.253796    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5" event={"ID":"1944156a-6d7a-4466-a481-a43e9fdbc0c8","Type":"ContainerStarted","Data":"71a07c5279a9171d9b93e742baf7551e949cc4c4f4ddb7b56d9b4a3a600f1829"}
Jul 14 16:36:26.261291 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:26.261257    2560 generic.go:356] "Generic (PLEG): container finished" podID="1944156a-6d7a-4466-a481-a43e9fdbc0c8" containerID="0315987618e037c966abbe7064e0f3c10ee02171b198ca64085d2d946a299e62" exitCode=0
Jul 14 16:36:26.261753 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:26.261343    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5" event={"ID":"1944156a-6d7a-4466-a481-a43e9fdbc0c8","Type":"ContainerDied","Data":"0315987618e037c966abbe7064e0f3c10ee02171b198ca64085d2d946a299e62"}
Jul 14 16:36:27.398650 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:27.398626    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5"
Jul 14 16:36:27.515591 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:27.515563    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1944156a-6d7a-4466-a481-a43e9fdbc0c8-maas-api-service-ca\") pod \"1944156a-6d7a-4466-a481-a43e9fdbc0c8\" (UID: \"1944156a-6d7a-4466-a481-a43e9fdbc0c8\") "
Jul 14 16:36:27.515947 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:27.515921    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1944156a-6d7a-4466-a481-a43e9fdbc0c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1944156a-6d7a-4466-a481-a43e9fdbc0c8" (UID: "1944156a-6d7a-4466-a481-a43e9fdbc0c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:36:27.616678 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:27.616634    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1944156a-6d7a-4466-a481-a43e9fdbc0c8-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:28.273830 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:28.273803    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5"
Jul 14 16:36:28.273830 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:28.273828    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5" event={"ID":"1944156a-6d7a-4466-a481-a43e9fdbc0c8","Type":"ContainerDied","Data":"71a07c5279a9171d9b93e742baf7551e949cc4c4f4ddb7b56d9b4a3a600f1829"}
Jul 14 16:36:28.274046 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:28.273868    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71a07c5279a9171d9b93e742baf7551e949cc4c4f4ddb7b56d9b4a3a600f1829"
Jul 14 16:36:35.358765 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.358730    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"]
Jul 14 16:36:35.359219 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.358983    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s" podUID="28835b46-101f-4c25-82af-711c6d1d9419" containerName="maas-api" containerID="cri-o://9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98" gracePeriod=30
Jul 14 16:36:35.414209 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.414170    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg"]
Jul 14 16:36:35.419417 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.419388    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-21b35e-29734110-zzxqg"]
Jul 14 16:36:35.607727 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.607704    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:36:35.689502 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.689411    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28835b46-101f-4c25-82af-711c6d1d9419-openshift-service-ca\") pod \"28835b46-101f-4c25-82af-711c6d1d9419\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") "
Jul 14 16:36:35.689502 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.689476    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28835b46-101f-4c25-82af-711c6d1d9419-maas-api-tls\") pod \"28835b46-101f-4c25-82af-711c6d1d9419\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") "
Jul 14 16:36:35.689669 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.689570    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t87hs\" (UniqueName: \"kubernetes.io/projected/28835b46-101f-4c25-82af-711c6d1d9419-kube-api-access-t87hs\") pod \"28835b46-101f-4c25-82af-711c6d1d9419\" (UID: \"28835b46-101f-4c25-82af-711c6d1d9419\") "
Jul 14 16:36:35.689794 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.689774    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28835b46-101f-4c25-82af-711c6d1d9419-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28835b46-101f-4c25-82af-711c6d1d9419" (UID: "28835b46-101f-4c25-82af-711c6d1d9419"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:36:35.689884 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.689865    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28835b46-101f-4c25-82af-711c6d1d9419-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:35.691687 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.691664    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28835b46-101f-4c25-82af-711c6d1d9419-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28835b46-101f-4c25-82af-711c6d1d9419" (UID: "28835b46-101f-4c25-82af-711c6d1d9419"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:36:35.691687 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.691661    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28835b46-101f-4c25-82af-711c6d1d9419-kube-api-access-t87hs" (OuterVolumeSpecName: "kube-api-access-t87hs") pod "28835b46-101f-4c25-82af-711c6d1d9419" (UID: "28835b46-101f-4c25-82af-711c6d1d9419"). InnerVolumeSpecName "kube-api-access-t87hs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:36:35.790894 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.790862    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t87hs\" (UniqueName: \"kubernetes.io/projected/28835b46-101f-4c25-82af-711c6d1d9419-kube-api-access-t87hs\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:35.790894 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:35.790891    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28835b46-101f-4c25-82af-711c6d1d9419-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:36.330123 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:36.330087    2560 generic.go:356] "Generic (PLEG): container finished" podID="28835b46-101f-4c25-82af-711c6d1d9419" containerID="9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98" exitCode=0
Jul 14 16:36:36.330297 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:36.330157    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"
Jul 14 16:36:36.330297 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:36.330171    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s" event={"ID":"28835b46-101f-4c25-82af-711c6d1d9419","Type":"ContainerDied","Data":"9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98"}
Jul 14 16:36:36.330297 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:36.330233    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s" event={"ID":"28835b46-101f-4c25-82af-711c6d1d9419","Type":"ContainerDied","Data":"f4c6b23b49d51c43132c9ebd2c088b264817f596f222751bbcea7b88ebd97c5b"}
Jul 14 16:36:36.330297 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:36.330264    2560 scope.go:117] "RemoveContainer" containerID="9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98"
Jul 14 16:36:36.340485 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:36.340468    2560 scope.go:117] "RemoveContainer" containerID="9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98"
Jul 14 16:36:36.340759 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:36:36.340737    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98\": container with ID starting with 9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98 not found: ID does not exist" containerID="9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98"
Jul 14 16:36:36.340834 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:36.340773    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98"} err="failed to get container status \"9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98\": rpc error: code = NotFound desc = could not find container \"9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98\": container with ID starting with 9ccb07fba827088707a4a30de8abcbc5897ddeb298236dc3a56d24265387ff98 not found: ID does not exist"
Jul 14 16:36:36.377692 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:36.377665    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"]
Jul 14 16:36:36.401833 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:36.401809    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-21b35e-67f4d77499-mfh5s"]
Jul 14 16:36:37.034119 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:37.034085    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28835b46-101f-4c25-82af-711c6d1d9419" path="/var/lib/kubelet/pods/28835b46-101f-4c25-82af-711c6d1d9419/volumes"
Jul 14 16:36:37.034427 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:37.034413    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4516f11f-7b0a-42e7-ae40-f945b83d58fd" path="/var/lib/kubelet/pods/4516f11f-7b0a-42e7-ae40-f945b83d58fd/volumes"
Jul 14 16:36:50.988943 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:50.988910    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n"]
Jul 14 16:36:50.989343 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:50.989305    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28835b46-101f-4c25-82af-711c6d1d9419" containerName="maas-api"
Jul 14 16:36:50.989343 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:50.989316    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="28835b46-101f-4c25-82af-711c6d1d9419" containerName="maas-api"
Jul 14 16:36:50.989343 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:50.989339    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1944156a-6d7a-4466-a481-a43e9fdbc0c8" containerName="revoke-keys"
Jul 14 16:36:50.989343 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:50.989344    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1944156a-6d7a-4466-a481-a43e9fdbc0c8" containerName="revoke-keys"
Jul 14 16:36:50.989498 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:50.989409    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1944156a-6d7a-4466-a481-a43e9fdbc0c8" containerName="revoke-keys"
Jul 14 16:36:50.989498 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:50.989420    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="28835b46-101f-4c25-82af-711c6d1d9419" containerName="maas-api"
Jul 14 16:36:50.992954 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:50.992931    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n"
Jul 14 16:36:50.996372 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:50.996338    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-62c9m\""
Jul 14 16:36:51.026895 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:51.026872    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff9a757-146d-45a0-a0fa-b6f0db2d59dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n\" (UID: \"7ff9a757-146d-45a0-a0fa-b6f0db2d59dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n"
Jul 14 16:36:51.066258 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:51.066224    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n"]
Jul 14 16:36:51.127332 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:51.127296    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff9a757-146d-45a0-a0fa-b6f0db2d59dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n\" (UID: \"7ff9a757-146d-45a0-a0fa-b6f0db2d59dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n"
Jul 14 16:36:51.127927 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:51.127910    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff9a757-146d-45a0-a0fa-b6f0db2d59dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n\" (UID: \"7ff9a757-146d-45a0-a0fa-b6f0db2d59dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n"
Jul 14 16:36:51.303492 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:51.303390    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n"
Jul 14 16:36:51.442548 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:51.442522    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n"]
Jul 14 16:36:51.444457 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:36:51.444426    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff9a757_146d_45a0_a0fa_b6f0db2d59dc.slice/crio-9c164cf2df3ee087f1f31fcabee6d66a0706013bb4238d4b11018a842bcfb71b WatchSource:0}: Error finding container 9c164cf2df3ee087f1f31fcabee6d66a0706013bb4238d4b11018a842bcfb71b: Status 404 returned error can't find the container with id 9c164cf2df3ee087f1f31fcabee6d66a0706013bb4238d4b11018a842bcfb71b
Jul 14 16:36:52.437056 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:52.436974    2560 generic.go:356] "Generic (PLEG): container finished" podID="7ff9a757-146d-45a0-a0fa-b6f0db2d59dc" containerID="51b5e207f770640691e0afed3a9cbd40aed88f79a1333ac6e93d3d55ab84f38f" exitCode=0
Jul 14 16:36:52.437056 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:52.437020    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n" event={"ID":"7ff9a757-146d-45a0-a0fa-b6f0db2d59dc","Type":"ContainerDied","Data":"51b5e207f770640691e0afed3a9cbd40aed88f79a1333ac6e93d3d55ab84f38f"}
Jul 14 16:36:52.437546 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:52.437089    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n" event={"ID":"7ff9a757-146d-45a0-a0fa-b6f0db2d59dc","Type":"ContainerStarted","Data":"9c164cf2df3ee087f1f31fcabee6d66a0706013bb4238d4b11018a842bcfb71b"}
Jul 14 16:36:53.571217 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:53.571192    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n"
Jul 14 16:36:53.651912 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:53.651888    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff9a757-146d-45a0-a0fa-b6f0db2d59dc-maas-api-service-ca\") pod \"7ff9a757-146d-45a0-a0fa-b6f0db2d59dc\" (UID: \"7ff9a757-146d-45a0-a0fa-b6f0db2d59dc\") "
Jul 14 16:36:53.652255 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:53.652233    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff9a757-146d-45a0-a0fa-b6f0db2d59dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ff9a757-146d-45a0-a0fa-b6f0db2d59dc" (UID: "7ff9a757-146d-45a0-a0fa-b6f0db2d59dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:36:53.753200 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:53.753124    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff9a757-146d-45a0-a0fa-b6f0db2d59dc-maas-api-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:36:54.449424 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:54.449395    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n"
Jul 14 16:36:54.449615 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:54.449434    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n" event={"ID":"7ff9a757-146d-45a0-a0fa-b6f0db2d59dc","Type":"ContainerDied","Data":"9c164cf2df3ee087f1f31fcabee6d66a0706013bb4238d4b11018a842bcfb71b"}
Jul 14 16:36:54.449615 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:54.449507    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c164cf2df3ee087f1f31fcabee6d66a0706013bb4238d4b11018a842bcfb71b"
Jul 14 16:36:57.189395 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:57.189364    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:36:57.201891 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:57.201868    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8c45s_ff82c9a1-cbcf-485b-9fe0-6e24d41da713/ovn-acl-logging/0.log"
Jul 14 16:36:57.362720 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:57.362695    2560 scope.go:117] "RemoveContainer" containerID="935fc39369d5d8ee2ac998c323bf2445899e6c986f187695b1cf1be0166cc93c"
Jul 14 16:36:57.370948 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:36:57.370930    2560 scope.go:117] "RemoveContainer" containerID="dcc7a62ffbc385d87d2d291347aa21286a4a1a91fa8821e0c58c0bc4da86e71a"
Jul 14 16:37:01.620569 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:01.620533    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"]
Jul 14 16:37:01.621030 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:01.620749    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn" podUID="2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5" containerName="maas-api" containerID="cri-o://e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25" gracePeriod=30
Jul 14 16:37:01.891306 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:01.891283    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:37:01.928390 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:01.928358    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-maas-api-tls\") pod \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") "
Jul 14 16:37:01.928588 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:01.928497    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh5vg\" (UniqueName: \"kubernetes.io/projected/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-kube-api-access-vh5vg\") pod \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") "
Jul 14 16:37:01.928588 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:01.928543    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-openshift-service-ca\") pod \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\" (UID: \"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5\") "
Jul 14 16:37:01.928874 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:01.928847    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5" (UID: "2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 16:37:01.930695 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:01.930669    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5" (UID: "2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 16:37:01.930801 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:01.930710    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-kube-api-access-vh5vg" (OuterVolumeSpecName: "kube-api-access-vh5vg") pod "2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5" (UID: "2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5"). InnerVolumeSpecName "kube-api-access-vh5vg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 16:37:02.029991 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.029964    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-maas-api-tls\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:37:02.029991 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.029987    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh5vg\" (UniqueName: \"kubernetes.io/projected/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-kube-api-access-vh5vg\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:37:02.029991 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.029997    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5-openshift-service-ca\") on node \"ip-10-0-135-20.ec2.internal\" DevicePath \"\""
Jul 14 16:37:02.492750 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.492719    2560 generic.go:356] "Generic (PLEG): container finished" podID="2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5" containerID="e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25" exitCode=0
Jul 14 16:37:02.492969 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.492763    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"
Jul 14 16:37:02.492969 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.492805    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn" event={"ID":"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5","Type":"ContainerDied","Data":"e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25"}
Jul 14 16:37:02.492969 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.492858    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn" event={"ID":"2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5","Type":"ContainerDied","Data":"6951205fab00cdc086e3d3abebe0441f7db586d8315d01fe83fd69504a511750"}
Jul 14 16:37:02.492969 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.492881    2560 scope.go:117] "RemoveContainer" containerID="e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25"
Jul 14 16:37:02.510148 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.510131    2560 scope.go:117] "RemoveContainer" containerID="e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25"
Jul 14 16:37:02.510395 ip-10-0-135-20 kubenswrapper[2560]: E0714 16:37:02.510375    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25\": container with ID starting with e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25 not found: ID does not exist" containerID="e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25"
Jul 14 16:37:02.510476 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.510404    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25"} err="failed to get container status \"e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25\": rpc error: code = NotFound desc = could not find container \"e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25\": container with ID starting with e5c25edd1be34840bf2d2f11dc4fa7eac37537bafa7c13ff1aa86cd92d391f25 not found: ID does not exist"
Jul 14 16:37:02.518766 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.518744    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"]
Jul 14 16:37:02.522885 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:02.522865    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e27c0-55b7cffdc-s9kmn"]
Jul 14 16:37:03.032521 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:03.032489    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5" path="/var/lib/kubelet/pods/2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5/volumes"
Jul 14 16:37:23.334278 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:23.334248    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7c4db64696-l4h24_bb9fd1e6-a308-4498-822f-2bd018f180b3/manager/0.log"
Jul 14 16:37:23.708130 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:23.708101    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7486689d9d-h8khl_8c5c6b30-088a-42a3-bedd-a386ae2657a7/copy-manifests/0.log"
Jul 14 16:37:23.716396 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:23.716353    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7486689d9d-h8khl_8c5c6b30-088a-42a3-bedd-a386ae2657a7/manager/0.log"
Jul 14 16:37:24.604269 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:24.604236    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734095-djtdj_294f8691-010b-4a61-b90f-6fb9fcb6481d/cleanup/0.log"
Jul 14 16:37:24.723322 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:24.723294    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734110-vw2mr_f42b0809-ca9d-49a2-862d-9812444e6483/cleanup/0.log"
Jul 14 16:37:24.839835 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:24.839811    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-7254e6a1-fdx7z_bcb65506-1da9-460b-b90b-926ad28a81de/revoke-keys/0.log"
Jul 14 16:37:25.072338 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:25.072265    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-d7e901-r8qds_1b1f4eba-1c6c-4a79-bb9f-65312909045f/revoke-keys/0.log"
Jul 14 16:37:25.190723 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:25.190701    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-c40588-5gcdf_c226d899-1aea-4db8-8b25-8c6ed7b8fce4/revoke-keys/0.log"
Jul 14 16:37:25.314803 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:25.314773    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-b8cdb4a9-7dlvb_4f759301-008b-4906-8f05-70329aac89ab/revoke-keys/0.log"
Jul 14 16:37:25.572498 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:25.572440    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-a4a505-782sp_82c61249-f6a9-4550-9c9e-b88e7e046b73/revoke-keys/0.log"
Jul 14 16:37:25.686723 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:25.686695    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-62755357-lkl4m_92e1a734-77e7-461b-b373-79259865d8e9/revoke-keys/0.log"
Jul 14 16:37:25.808523 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:25.808494    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-ad3a3819-87wtv_e36b08e2-cdf0-4f48-9869-f5fd6e72b78b/revoke-keys/0.log"
Jul 14 16:37:25.924811 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:25.924780    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-d453a59f-6sktr_9a798393-5667-44d8-afba-9d430b12eef6/revoke-keys/0.log"
Jul 14 16:37:26.042390 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:26.042363    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-21b35e-scfj5_1944156a-6d7a-4466-a481-a43e9fdbc0c8/revoke-keys/0.log"
Jul 14 16:37:26.159835 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:26.159802    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-4e27c0-nvb5n_7ff9a757-146d-45a0-a0fa-b6f0db2d59dc/revoke-keys/0.log"
Jul 14 16:37:28.032168 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:28.032110    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8jwwj_e4ee7d3a-ab03-47c7-851f-93600b3ff276/manager/0.log"
Jul 14 16:37:28.144333 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:28.144307    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-z72nv_bfaff471-4b4c-4986-a88d-7b94d15f86da/kuadrant-console-plugin/0.log"
Jul 14 16:37:28.661015 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:28.660978    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-nv6pb_f93a3446-61bb-4619-9c7e-96b51398ff73/manager/0.log"
Jul 14 16:37:29.279764 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:29.279735    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85568bb86c-99mrc_9230b14d-42a8-4647-aba8-10bd740658c0/kube-auth-proxy/0.log"
Jul 14 16:37:29.653224 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:29.653193    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-j5frn_fe798d69-e937-4478-8208-73f94dac44f7/payload-pre-processing/0.log"
Jul 14 16:37:30.251177 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:30.251148    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-xpfg7_6062ac05-4f49-499b-b257-689e49d74d4b/main/0.log"
Jul 14 16:37:38.776607 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:38.776579    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8674z_0b801942-157e-4913-98a7-44d9440b3257/global-pull-secret-syncer/0.log"
Jul 14 16:37:39.015517 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:39.015488    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lx86m_f5ac6baa-690d-477c-8090-199341500f68/konnectivity-agent/0.log"
Jul 14 16:37:39.075039 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:39.074964    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-20.ec2.internal_7b65546031303e49503806457ed26462/haproxy/0.log"
Jul 14 16:37:44.268140 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:44.268055    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8jwwj_e4ee7d3a-ab03-47c7-851f-93600b3ff276/manager/0.log"
Jul 14 16:37:44.299192 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:44.299166    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-z72nv_bfaff471-4b4c-4986-a88d-7b94d15f86da/kuadrant-console-plugin/0.log"
Jul 14 16:37:44.570268 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:44.570191    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-nv6pb_f93a3446-61bb-4619-9c7e-96b51398ff73/manager/0.log"
Jul 14 16:37:46.278831 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.278797    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a75ec2-b796-4898-876e-4a08aef3264f/alertmanager/0.log"
Jul 14 16:37:46.320198 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.320146    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a75ec2-b796-4898-876e-4a08aef3264f/config-reloader/0.log"
Jul 14 16:37:46.363258 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.363234    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a75ec2-b796-4898-876e-4a08aef3264f/kube-rbac-proxy-web/0.log"
Jul 14 16:37:46.406704 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.406673    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a75ec2-b796-4898-876e-4a08aef3264f/kube-rbac-proxy/0.log"
Jul 14 16:37:46.439018 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.438996    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a75ec2-b796-4898-876e-4a08aef3264f/kube-rbac-proxy-metric/0.log"
Jul 14 16:37:46.480130 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.480105    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a75ec2-b796-4898-876e-4a08aef3264f/prom-label-proxy/0.log"
Jul 14 16:37:46.518216 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.518146    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93a75ec2-b796-4898-876e-4a08aef3264f/init-config-reloader/0.log"
Jul 14 16:37:46.636160 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.636130    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-prfgb_c827943c-e597-4cd3-8964-a55060096e6d/kube-state-metrics/0.log"
Jul 14 16:37:46.666853 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.666819    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-prfgb_c827943c-e597-4cd3-8964-a55060096e6d/kube-rbac-proxy-main/0.log"
Jul 14 16:37:46.702324 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.702299    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-prfgb_c827943c-e597-4cd3-8964-a55060096e6d/kube-rbac-proxy-self/0.log"
Jul 14 16:37:46.741372 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.741344    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c464d7cf4-4w7b2_ee471a45-f22c-4773-96b7-c96d1da35cee/metrics-server/0.log"
Jul 14 16:37:46.789043 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.788977    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-9f57r_79315669-ca17-45a4-9bb9-303e739eb8dd/monitoring-plugin/0.log"
Jul 14 16:37:46.837767 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.837747    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g8z6p_8ee18d4c-17a6-424d-9a79-5e01177de242/node-exporter/0.log"
Jul 14 16:37:46.872607 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.872584    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g8z6p_8ee18d4c-17a6-424d-9a79-5e01177de242/kube-rbac-proxy/0.log"
Jul 14 16:37:46.918232 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:46.918211    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g8z6p_8ee18d4c-17a6-424d-9a79-5e01177de242/init-textfile/0.log"
Jul 14 16:37:47.260796 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.260694    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-bzc2t_8586edde-9643-4c5f-932f-0597d5afd316/kube-rbac-proxy-main/0.log"
Jul 14 16:37:47.306591 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.306561    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-bzc2t_8586edde-9643-4c5f-932f-0597d5afd316/kube-rbac-proxy-self/0.log"
Jul 14 16:37:47.357638 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.357615    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-bzc2t_8586edde-9643-4c5f-932f-0597d5afd316/openshift-state-metrics/0.log"
Jul 14 16:37:47.398035 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.398000    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_92387b00-938f-451a-8aac-776a13df9d61/prometheus/0.log"
Jul 14 16:37:47.422248 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.422205    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_92387b00-938f-451a-8aac-776a13df9d61/config-reloader/0.log"
Jul 14 16:37:47.455417 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.455386    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_92387b00-938f-451a-8aac-776a13df9d61/thanos-sidecar/0.log"
Jul 14 16:37:47.482968 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.482944    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_92387b00-938f-451a-8aac-776a13df9d61/kube-rbac-proxy-web/0.log"
Jul 14 16:37:47.515321 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.515283    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_92387b00-938f-451a-8aac-776a13df9d61/kube-rbac-proxy/0.log"
Jul 14 16:37:47.547869 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.547849    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_92387b00-938f-451a-8aac-776a13df9d61/kube-rbac-proxy-thanos/0.log"
Jul 14 16:37:47.597677 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.597650    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_92387b00-938f-451a-8aac-776a13df9d61/init-config-reloader/0.log"
Jul 14 16:37:47.639572 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.639433    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-fstjl_43282f36-5709-462b-9d94-117b37c45a74/prometheus-operator/0.log"
Jul 14 16:37:47.721003 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.720953    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-fstjl_43282f36-5709-462b-9d94-117b37c45a74/kube-rbac-proxy/0.log"
Jul 14 16:37:47.828268 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.828238    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b54b766fc-c5k2m_1cc47fc7-d572-4281-9e26-71c1ef812bea/telemeter-client/0.log"
Jul 14 16:37:47.856042 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.856020    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b54b766fc-c5k2m_1cc47fc7-d572-4281-9e26-71c1ef812bea/reload/0.log"
Jul 14 16:37:47.883876 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.883854    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b54b766fc-c5k2m_1cc47fc7-d572-4281-9e26-71c1ef812bea/kube-rbac-proxy/0.log"
Jul 14 16:37:47.924357 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.924296    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5965d957f-58mld_812da01e-0d77-451a-8942-eb32e0f0728a/thanos-query/0.log"
Jul 14 16:37:47.955494 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.955471    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5965d957f-58mld_812da01e-0d77-451a-8942-eb32e0f0728a/kube-rbac-proxy-web/0.log"
Jul 14 16:37:47.981075 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:47.981056    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5965d957f-58mld_812da01e-0d77-451a-8942-eb32e0f0728a/kube-rbac-proxy/0.log"
Jul 14 16:37:48.007269 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:48.007242    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5965d957f-58mld_812da01e-0d77-451a-8942-eb32e0f0728a/prom-label-proxy/0.log"
Jul 14 16:37:48.031462 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:48.031418    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5965d957f-58mld_812da01e-0d77-451a-8942-eb32e0f0728a/kube-rbac-proxy-rules/0.log"
Jul 14 16:37:48.055275 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:48.055245    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5965d957f-58mld_812da01e-0d77-451a-8942-eb32e0f0728a/kube-rbac-proxy-metrics/0.log"
Jul 14 16:37:50.179963 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.179937    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7fbb5fd7-x9mp7_8058c125-7ec1-4eaf-b480-d979fd3c394a/console/0.log"
Jul 14 16:37:50.420101 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.420064    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"]
Jul 14 16:37:50.420553 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.420534    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ff9a757-146d-45a0-a0fa-b6f0db2d59dc" containerName="revoke-keys"
Jul 14 16:37:50.420553 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.420552    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ff9a757-146d-45a0-a0fa-b6f0db2d59dc" containerName="revoke-keys"
Jul 14 16:37:50.420765 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.420572    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5" containerName="maas-api"
Jul 14 16:37:50.420765 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.420577    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5" containerName="maas-api"
Jul 14 16:37:50.420765 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.420655    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e8dbf7a-6d96-48f7-88e5-fdaae81d29b5" containerName="maas-api"
Jul 14 16:37:50.420765 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.420672    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ff9a757-146d-45a0-a0fa-b6f0db2d59dc" containerName="revoke-keys"
Jul 14 16:37:50.424073 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.424056    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.426345 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.426315    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wmsrf\"/\"kube-root-ca.crt\""
Jul 14 16:37:50.426541 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.426320    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wmsrf\"/\"openshift-service-ca.crt\""
Jul 14 16:37:50.426541 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.426325    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wmsrf\"/\"default-dockercfg-pvx4s\""
Jul 14 16:37:50.434000 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.433978    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"]
Jul 14 16:37:50.468235 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.468204    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-sys\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.468235 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.468240    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-podres\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.468408 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.468321    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-lib-modules\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.468408 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.468364    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp29p\" (UniqueName: \"kubernetes.io/projected/c02c94fc-3344-473f-b2fb-bbd8533a102d-kube-api-access-kp29p\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.468513 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.468418    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-proc\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.569159 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.569126    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-lib-modules\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.569326 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.569177    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp29p\" (UniqueName: \"kubernetes.io/projected/c02c94fc-3344-473f-b2fb-bbd8533a102d-kube-api-access-kp29p\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.569326 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.569247    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-proc\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.569326 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.569282    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-sys\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.569326 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.569310    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-podres\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.569532 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.569342    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-proc\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.569532 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.569311    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-lib-modules\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.569532 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.569396    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-sys\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.569532 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.569423    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c02c94fc-3344-473f-b2fb-bbd8533a102d-podres\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.581586 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.581567    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp29p\" (UniqueName: \"kubernetes.io/projected/c02c94fc-3344-473f-b2fb-bbd8533a102d-kube-api-access-kp29p\") pod \"perf-node-gather-daemonset-z45g6\" (UID: \"c02c94fc-3344-473f-b2fb-bbd8533a102d\") " pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.735894 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.735799    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:50.874177 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:50.874149    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"]
Jul 14 16:37:50.875989 ip-10-0-135-20 kubenswrapper[2560]: W0714 16:37:50.875961    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc02c94fc_3344_473f_b2fb_bbd8533a102d.slice/crio-5a3f8cb357f438185ad732d49506841541fa743f976fd67664db8d9ec11207c3 WatchSource:0}: Error finding container 5a3f8cb357f438185ad732d49506841541fa743f976fd67664db8d9ec11207c3: Status 404 returned error can't find the container with id 5a3f8cb357f438185ad732d49506841541fa743f976fd67664db8d9ec11207c3
Jul 14 16:37:51.761767 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:51.761731    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6" event={"ID":"c02c94fc-3344-473f-b2fb-bbd8533a102d","Type":"ContainerStarted","Data":"8ad186060f1e73b05a457ac31eaa856ae1aef9d98a4dbdba0d2f9cf6507eab98"}
Jul 14 16:37:51.762183 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:51.761794    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6" event={"ID":"c02c94fc-3344-473f-b2fb-bbd8533a102d","Type":"ContainerStarted","Data":"5a3f8cb357f438185ad732d49506841541fa743f976fd67664db8d9ec11207c3"}
Jul 14 16:37:51.762183 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:51.761823    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:51.783055 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:51.783002    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6" podStartSLOduration=1.782987312 podStartE2EDuration="1.782987312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 16:37:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 16:37:51.782259523 +0000 UTC m=+3055.301626954" watchObservedRunningTime="2026-07-14 16:37:51.782987312 +0000 UTC m=+3055.302354733"
Jul 14 16:37:52.003853 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:52.003822    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8mzm5_ec667542-ec92-4054-a5d5-849cfe374814/dns/0.log"
Jul 14 16:37:52.032156 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:52.032097    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8mzm5_ec667542-ec92-4054-a5d5-849cfe374814/kube-rbac-proxy/0.log"
Jul 14 16:37:52.235714 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:52.235688    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mtpnp_0cee7827-8938-4376-8ba2-65a519b89996/dns-node-resolver/0.log"
Jul 14 16:37:52.897931 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:52.897888    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sbkld_c4921108-78c3-47e5-9965-2882382685de/node-ca/0.log"
Jul 14 16:37:54.087095 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:54.087063    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85568bb86c-99mrc_9230b14d-42a8-4647-aba8-10bd740658c0/kube-auth-proxy/0.log"
Jul 14 16:37:54.206998 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:54.206969    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-j5frn_fe798d69-e937-4478-8208-73f94dac44f7/payload-pre-processing/0.log"
Jul 14 16:37:55.009669 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:55.009636    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nwpfv_e6b29cdc-5f48-48ad-b97a-42a968c2ce7d/serve-healthcheck-canary/0.log"
Jul 14 16:37:55.716703 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:55.716678    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zkc2d_e8c6cfcb-efe3-4838-b2d9-614bd266711a/kube-rbac-proxy/0.log"
Jul 14 16:37:55.740857 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:55.740832    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zkc2d_e8c6cfcb-efe3-4838-b2d9-614bd266711a/exporter/0.log"
Jul 14 16:37:55.767102 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:55.767077    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zkc2d_e8c6cfcb-efe3-4838-b2d9-614bd266711a/extractor/0.log"
Jul 14 16:37:57.777376 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:57.777304    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wmsrf/perf-node-gather-daemonset-z45g6"
Jul 14 16:37:58.281727 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:58.281675    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7c4db64696-l4h24_bb9fd1e6-a308-4498-822f-2bd018f180b3/manager/0.log"
Jul 14 16:37:58.369286 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:58.369252    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85568bb86c-99mrc_9230b14d-42a8-4647-aba8-10bd740658c0/kube-auth-proxy/0.log"
Jul 14 16:37:58.404656 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:58.404636    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7486689d9d-h8khl_8c5c6b30-088a-42a3-bedd-a386ae2657a7/manager/0.log"
Jul 14 16:37:58.441943 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:58.441907    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7486689d9d-h8khl_8c5c6b30-088a-42a3-bedd-a386ae2657a7/copy-manifests/0.log"
Jul 14 16:37:58.460149 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:37:58.460122    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-j5frn_fe798d69-e937-4478-8208-73f94dac44f7/payload-pre-processing/0.log"
Jul 14 16:38:00.058292 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:38:00.058217    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85568bb86c-99mrc_9230b14d-42a8-4647-aba8-10bd740658c0/kube-auth-proxy/0.log"
Jul 14 16:38:00.153566 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:38:00.153538    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-658b99d874-hlsdr_8a453c4c-a2ac-49e9-b663-59217c2fe45c/manager/0.log"
Jul 14 16:38:00.180351 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:38:00.180327    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-j5frn_fe798d69-e937-4478-8208-73f94dac44f7/payload-pre-processing/0.log"
Jul 14 16:38:00.192764 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:38:00.192744    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rtfw6_917b405c-b5f6-49ff-be48-41b30189b2c6/openshift-lws-operator/0.log"
Jul 14 16:38:07.251402 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:38:07.251369    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g9l8z_77a7601c-9b76-4736-94a3-0f9007c6c221/kube-multus-additional-cni-plugins/0.log"
Jul 14 16:38:07.279943 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:38:07.279917    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g9l8z_77a7601c-9b76-4736-94a3-0f9007c6c221/egress-router-binary-copy/0.log"
Jul 14 16:38:07.308786 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:38:07.308756    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g9l8z_77a7601c-9b76-4736-94a3-0f9007c6c221/cni-plugins/0.log"
Jul 14 16:38:07.342473 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:38:07.342423    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g9l8z_77a7601c-9b76-4736-94a3-0f9007c6c221/bond-cni-plugin/0.log"
Jul 14 16:38:07.370873 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:38:07.370849    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g9l8z_77a7601c-9b76-4736-94a3-0f9007c6c221/routeoverride-cni/0.log"
Jul 14 16:38:07.400281 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:38:07.400256    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g9l8z_77a7601c-9b76-4736-94a3-0f9007c6c221/whereabouts-cni-bincopy/0.log"
Jul 14 16:38:07.426895 ip-10-0-135-20 kubenswrapper[2560]: I0714 16:38:07.426869    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g9l8z_77a7601c-9b76-4736-94a3-0f9007c6c221/whereabouts-cni/0.log"