Jul 08 23:03:00.955828 ip-10-0-131-54 systemd[1]: Starting Kubernetes Kubelet...
Jul 08 23:03:01.451314 ip-10-0-131-54 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 23:03:01.451314 ip-10-0-131-54 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 08 23:03:01.451314 ip-10-0-131-54 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 23:03:01.451314 ip-10-0-131-54 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 08 23:03:01.451314 ip-10-0-131-54 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 23:03:01.452352 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.452260    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 08 23:03:01.459187 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459164    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 23:03:01.459187 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459183    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 23:03:01.459187 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459189    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 23:03:01.459187 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459192    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 23:03:01.459187 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459196    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459200    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459203    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459206    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459209    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459211    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459214    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459217    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459220    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459223    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459226    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459229    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459231    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459234    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459237    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459239    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459242    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459245    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459247    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459250    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 23:03:01.459379 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459252    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459255    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459257    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459260    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459262    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459265    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459268    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459270    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459273    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459275    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459278    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459280    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459283    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459286    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459290    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459293    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459295    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459298    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459300    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459303    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 23:03:01.459866 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459305    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459308    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459310    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459313    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459316    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459318    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459321    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459323    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459326    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459329    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459331    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459334    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459337    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459340    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459343    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459346    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459348    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459352    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459356    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459360    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 23:03:01.460352 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459362    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459365    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459367    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459370    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459372    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459376    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459379    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459384    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459387    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459391    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459394    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459397    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459400    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459403    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459406    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459409    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459411    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459414    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459417    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 23:03:01.460831 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459420    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459423    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459426    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459428    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459431    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459433    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459436    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459438    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459441    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459444    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459446    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459449    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459451    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459454    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459456    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.459459    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460192    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460200    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460203    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460206    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 23:03:01.461314 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460208    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460211    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460224    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460227    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460230    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460233    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460235    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460238    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460241    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460244    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460247    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460249    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460252    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460256    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460258    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460261    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460264    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460266    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460269    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460271    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 23:03:01.461803 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460276    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460279    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460282    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460285    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460288    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460291    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460294    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460298    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460301    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460304    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460308    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460311    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460314    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460317    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460319    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460322    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460324    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460327    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460330    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460332    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 23:03:01.462324 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460335    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460337    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460340    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460342    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460345    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460347    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460350    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460353    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460355    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460358    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460361    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460363    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460366    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460368    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460371    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460373    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460376    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460378    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460381    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460384    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 23:03:01.462814 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460386    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460389    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460391    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460395    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460397    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460400    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460402    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460405    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460408    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460410    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460413    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460415    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460417    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460420    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460422    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460425    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460427    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460430    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460432    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 23:03:01.463292 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460435    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460438    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460440    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460443    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460445    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460449    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460452    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460454    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460458    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460461    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460479    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460483    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460486    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460488    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460491    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460495    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460874    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460878    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460881    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 23:03:01.463759 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460884    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460887    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460891    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460893    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460896    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460899    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460902    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460905    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460908    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460911    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460914    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460916    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460919    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460922    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460925    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460927    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460930    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460932    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460935    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460937    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 23:03:01.464210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460939    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460942    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460944    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460947    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460949    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460952    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460955    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460958    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460960    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460963    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460965    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460968    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460970    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460973    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460975    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460978    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460980    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460983    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460985    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 23:03:01.464689 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460994    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.460998    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461000    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461003    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461005    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461008    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461010    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461013    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461016    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461018    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461020    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461023    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461026    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461028    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461032    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461035    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461038    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461041    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461044    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 23:03:01.465174 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461047    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461049    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461052    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461055    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461060    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461063    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461066    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461069    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461072    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461074    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461077    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461079    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461082    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461084    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461087    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461089    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461092    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461095    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461097    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461100    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 23:03:01.465640 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461102    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461105    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461108    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461110    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461113    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461115    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461118    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461120    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461123    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461125    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461127    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461130    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461133    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461135    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461138    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461141    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461143    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461146    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461216    2560 flags.go:64] FLAG: --address="0.0.0.0"
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461224    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461231    2560 flags.go:64] FLAG: --anonymous-auth="true"
Jul 08 23:03:01.466110 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461235    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461239    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461242    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461247    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461251    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461254    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461257    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461260    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461264    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461267    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461270    2560 flags.go:64] FLAG: --cgroup-root=""
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461273    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461276    2560 flags.go:64] FLAG: --client-ca-file=""
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461283    2560 flags.go:64] FLAG: --cloud-provider="external"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461286    2560 flags.go:64] FLAG: --cluster-dns="[]"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461290    2560 flags.go:64] FLAG: --cluster-domain=""
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461293    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461296    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461299    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461302    2560 flags.go:64] FLAG: --container-log-max-files="5"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461305    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461308    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461311    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 08 23:03:01.466717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461314    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461318    2560 flags.go:64] FLAG: --contention-profiling="false"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461323    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461326    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461329    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461332    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461338    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461341    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461344    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461347    2560 flags.go:64] FLAG: --enable-load-reader="false"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461350    2560 flags.go:64] FLAG: --enable-server="true"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461353    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461357    2560 flags.go:64] FLAG: --event-burst="100"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461360    2560 flags.go:64] FLAG: --event-qps="50"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461363    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461366    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461369    2560 flags.go:64] FLAG: --eviction-hard=""
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461373    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461376    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461379    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461382    2560 flags.go:64] FLAG: --eviction-soft=""
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461384    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461387    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461391    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461394    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 08 23:03:01.467775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461396    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461399    2560 flags.go:64] FLAG: --fail-swap-on="true"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461402    2560 flags.go:64] FLAG: --feature-gates=""
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461405    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461408    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461411    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461414    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461417    2560 flags.go:64] FLAG: --healthz-port="10248"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461420    2560 flags.go:64] FLAG: --help="false"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461424    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461427    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461430    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461433    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461436    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461439    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461442    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461445    2560 flags.go:64] FLAG: --image-service-endpoint=""
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461448    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461451    2560 flags.go:64] FLAG: --kube-api-burst="100"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461453    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461456    2560 flags.go:64] FLAG: --kube-api-qps="50"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461459    2560 flags.go:64] FLAG: --kube-reserved=""
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461462    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461481    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 08 23:03:01.469008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461485    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461487    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461490    2560 flags.go:64] FLAG: --lock-file=""
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461493    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461496    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461499    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461504    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461508    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461510    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461513    2560 flags.go:64] FLAG: --logging-format="text"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461516    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461519    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461522    2560 flags.go:64] FLAG: --manifest-url=""
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461527    2560 flags.go:64] FLAG: --manifest-url-header=""
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461531    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461534    2560 flags.go:64] FLAG: --max-open-files="1000000"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461538    2560 flags.go:64] FLAG: --max-pods="110"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461543    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461545    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461548    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461551    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461554    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461557    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461559    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461566    2560 flags.go:64] FLAG: --node-status-max-images="50"
Jul 08 23:03:01.469658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461569    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461572    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461575    2560 flags.go:64] FLAG: --pod-cidr=""
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461578    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461583    2560 flags.go:64] FLAG: --pod-manifest-path=""
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461586    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461589    2560 flags.go:64] FLAG: --pods-per-core="0"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461592    2560 flags.go:64] FLAG: --port="10250"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461595    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461598    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a6c51bf4e89025f5"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461601    2560 flags.go:64] FLAG: --qos-reserved=""
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461604    2560 flags.go:64] FLAG: --read-only-port="10255"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461607    2560 flags.go:64] FLAG: --register-node="true"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461610    2560 flags.go:64] FLAG: --register-with-taints=""
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461614    2560 flags.go:64] FLAG: --registry-burst="10"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461617    2560 flags.go:64] FLAG: --registry-qps="5"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461620    2560 flags.go:64] FLAG: --reserved-cpus=""
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461623    2560 flags.go:64] FLAG: --reserved-memory=""
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461626    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461629    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461634    2560 flags.go:64] FLAG: --rotate-certificates="false"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461636    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461639    2560 flags.go:64] FLAG: --runonce="false"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461642    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461645    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 08 23:03:01.470295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461650    2560 flags.go:64] FLAG: --seccomp-default="false"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461653    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461656    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461658    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461662    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461665    2560 flags.go:64] FLAG: --storage-driver-password="root"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461668    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461671    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461674    2560 flags.go:64] FLAG: --storage-driver-user="root"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461676    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461679    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461682    2560 flags.go:64] FLAG: --system-cgroups=""
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461685    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461690    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461693    2560 flags.go:64] FLAG: --tls-cert-file=""
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461695    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461699    2560 flags.go:64] FLAG: --tls-min-version=""
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461702    2560 flags.go:64] FLAG: --tls-private-key-file=""
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461704    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461707    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461710    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461713    2560 flags.go:64] FLAG: --v="2"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461717    2560 flags.go:64] FLAG: --version="false"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461721    2560 flags.go:64] FLAG: --vmodule=""
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461725    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 08 23:03:01.470984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.461728    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461843    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461850    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461854    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461856    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461859    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461862    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461866    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461869    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461873    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461875    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461878    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461880    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461883    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461885    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461888    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461890    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461893    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461895    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461898    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 23:03:01.471701 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461901    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461903    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461906    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461908    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461911    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461913    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461916    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461919    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461921    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461924    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461926    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461929    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461932    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461934    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461939    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461942    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461944    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461946    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461949    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461954    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 23:03:01.472215 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461956    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461959    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461963    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461966    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461969    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461972    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461974    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461976    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461979    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461983    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461986    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461989    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461993    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461995    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.461998    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462001    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462004    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462007    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462010    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 23:03:01.472773 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462013    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462016    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462019    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462022    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462024    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462028    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462031    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462035    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462038    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462040    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462043    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462046    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462050    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462053    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462055    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462059    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462063    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462065    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462068    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 23:03:01.473258 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462071    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462074    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462076    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462079    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462081    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462084    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462086    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462089    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462091    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462094    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462096    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462099    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462101    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462104    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462106    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462109    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462111    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462114    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462117    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462120    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 23:03:01.473777 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462124    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.462126    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.462131    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.470333    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.470351    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.472104    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.472112    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472178    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472184    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472187    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472191    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472194    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472197    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472200    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472203    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 23:03:01.474288 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472206    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472210    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472214    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472218    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472222    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472225    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472228    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472232    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472235    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472238    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472241    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472244    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472247    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472249    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472252    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472255    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472258    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472261    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472264    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 23:03:01.474698 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472267    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472270    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472274    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472277    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472279    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472282    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472285    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472288    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472292    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472294    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472297    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472300    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472303    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472305    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472308    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472311    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472313    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472316    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472319    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472322    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 23:03:01.475198 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472325    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472328    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472331    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472334    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472336    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472339    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472341    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472344    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472347    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472350    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472352    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472355    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472357    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472360    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472364    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472368    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472371    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472374    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472376    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 23:03:01.475787 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472379    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472382    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472385    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472387    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472391    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472394    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472396    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472399    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472402    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472404    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472407    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472410    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472412    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472415    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472418    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472421    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472425    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472428    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472431    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472434    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 23:03:01.476261 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472437    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472440    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472442    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472445    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472448    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472450    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472454    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472457    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472460    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472463    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472466    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472469    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472473    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.472479    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 23:03:01.476795 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472641    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472646    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472651    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472656    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472670    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472674    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472677    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472680    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472683    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472686    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472689    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472692    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472695    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472698    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472701    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472704    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472709    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472713    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472716    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 23:03:01.477172 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472719    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472721    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472724    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472727    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472729    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472733    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472736    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472739    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472742    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472746    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472749    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472751    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472754    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472757    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472760    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472762    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472765    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472768    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472770    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472773    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472776    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 23:03:01.477703 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472778    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472781    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472784    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472788    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472790    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472793    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472796    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472799    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472801    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472804    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472807    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472809    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472812    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472815    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472817    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472820    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472822    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472825    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472828    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 23:03:01.478235 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472831    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472834    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472837    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472840    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472843    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472845    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472850    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472853    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472856    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472859    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472862    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472865    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472868    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472870    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472873    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472876    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472878    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472881    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472884    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 23:03:01.478751 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472887    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472890    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472892    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472895    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472898    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472900    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472903    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472906    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472908    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472911    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472913    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472916    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472919    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472922    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472925    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472927    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472931    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472934    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472936    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472939    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 23:03:01.479277 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:01.472941    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 23:03:01.479901 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.472946    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 23:03:01.479901 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.473072    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 08 23:03:01.479901 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.476330    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 08 23:03:01.479901 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.478042    2560 server.go:1019] "Starting client certificate rotation"
Jul 08 23:03:01.479901 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.478162    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 23:03:01.479901 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.478196    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 23:03:01.481489 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.481472    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 23:03:01.483352 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.483281    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 23:03:01.496133 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.496103    2560 log.go:25] "Validated CRI v1 runtime API"
Jul 08 23:03:01.498853 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.498839    2560 log.go:25] "Validated CRI v1 image API"
Jul 08 23:03:01.500389 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.500374    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 08 23:03:01.504416 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.504396    2560 fs.go:135] Filesystem UUIDs: map[71dba72e-0157-4581-969f-ca7000fb3a7f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c726a5dc-5158-4e20-990d-9973c165b9b7:/dev/nvme0n1p4]
Jul 08 23:03:01.504503 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.504416    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 08 23:03:01.509375 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.509354    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 23:03:01.510394 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.510276    2560 manager.go:216] Machine: {Timestamp:2026-07-08 23:03:01.508883261 +0000 UTC m=+0.438917229 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3063077 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2976877b6772903bd3590a2dc8c765 SystemUUID:ec297687-7b67-7290-3bd3-590a2dc8c765 BootID:d8201c0d-b56d-420f-b4e1-81dfd377ec52 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5a:aa:7b:8d:d1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5a:aa:7b:8d:d1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:40:cd:76:d7:6f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 08 23:03:01.510394 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.510388    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 08 23:03:01.510523 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.510511    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 08 23:03:01.513398 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.513370    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 08 23:03:01.513581 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.513402    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-54.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 08 23:03:01.515083 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.515072    2560 topology_manager.go:138] "Creating topology manager with none policy"
Jul 08 23:03:01.515117 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.515085    2560 container_manager_linux.go:309] "Creating device plugin manager"
Jul 08 23:03:01.515117 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.515094    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 23:03:01.515898 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.515888    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 23:03:01.517121 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.517110    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 08 23:03:01.520717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.520705    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 23:03:01.520846 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.520836    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 08 23:03:01.520879 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.520868    2560 kubelet.go:486] "Attempting to sync node with API server"
Jul 08 23:03:01.520908 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.520879    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 08 23:03:01.520908 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.520890    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 08 23:03:01.520908 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.520900    2560 kubelet.go:398] "Adding apiserver pod source"
Jul 08 23:03:01.520992 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.520915    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 08 23:03:01.521331 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.521318    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 23:03:01.521395 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.521339    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 23:03:01.523340 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.523319    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 08 23:03:01.525098 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.525083    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 08 23:03:01.525157 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.525115    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 08 23:03:01.526360 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526348    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 08 23:03:01.526426 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526365    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 08 23:03:01.526426 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526391    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 08 23:03:01.526426 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526398    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 08 23:03:01.526426 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526409    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 08 23:03:01.526426 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526415    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 08 23:03:01.526426 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526423    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 08 23:03:01.526426 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526430    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 08 23:03:01.526642 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526437    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 08 23:03:01.526642 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526443    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 08 23:03:01.526642 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526457    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 08 23:03:01.526642 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.526464    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 08 23:03:01.527416 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.527403    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 08 23:03:01.527416 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.527417    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 08 23:03:01.530757 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.530737    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ctfrv"
Jul 08 23:03:01.530757 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.530743    2560 server.go:1268] "Started kubelet"
Jul 08 23:03:01.530923 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.530867    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 08 23:03:01.531191 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.531072    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 08 23:03:01.531191 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.531152    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 08 23:03:01.531819 ip-10-0-131-54 systemd[1]: Started Kubernetes Kubelet.
Jul 08 23:03:01.532656 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.532413    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-54.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 08 23:03:01.532656 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.532424    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 08 23:03:01.532656 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.532516    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 08 23:03:01.533662 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.533644    2560 server.go:310] "Adding debug handlers to kubelet server"
Jul 08 23:03:01.535279 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.535263    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 08 23:03:01.539749 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.536957    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-54.ec2.internal.18c072e548a76006  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-54.ec2.internal,UID:ip-10-0-131-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-54.ec2.internal,},FirstTimestamp:2026-07-08 23:03:01.53072231 +0000 UTC m=+0.460756278,LastTimestamp:2026-07-08 23:03:01.53072231 +0000 UTC m=+0.460756278,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-54.ec2.internal,}"
Jul 08 23:03:01.539848 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.539774    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ctfrv"
Jul 08 23:03:01.540894 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.540848    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 08 23:03:01.553547 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.553524    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 08 23:03:01.553643 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.553528    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 08 23:03:01.553643 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.553634    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-54.ec2.internal\" not found"
Jul 08 23:03:01.554308 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554293    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 08 23:03:01.554400 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554293    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 08 23:03:01.554400 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554381    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 08 23:03:01.554541 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554463    2560 reconstruct.go:97] "Volume reconstruction finished"
Jul 08 23:03:01.554541 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554491    2560 reconciler.go:29] "Reconciler: start to sync state"
Jul 08 23:03:01.554541 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554518    2560 factory.go:55] Registering systemd factory
Jul 08 23:03:01.554541 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554537    2560 factory.go:223] Registration of the systemd container factory successfully
Jul 08 23:03:01.554798 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554783    2560 factory.go:153] Registering CRI-O factory
Jul 08 23:03:01.554845 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554802    2560 factory.go:223] Registration of the crio container factory successfully
Jul 08 23:03:01.554892 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554863    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 08 23:03:01.554892 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554889    2560 factory.go:103] Registering Raw factory
Jul 08 23:03:01.554958 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.554907    2560 manager.go:1196] Started watching for new ooms in manager
Jul 08 23:03:01.555406 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.555394    2560 manager.go:349] Starting recovery of all containers
Jul 08 23:03:01.556637 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.556613    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 08 23:03:01.556720 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.556694    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 08 23:03:01.565720 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.565699    2560 manager.go:354] Recovery completed
Jul 08 23:03:01.569701 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.569686    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 23:03:01.571495 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.571481    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 23:03:01.571588 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.571514    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 23:03:01.571588 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.571529    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 23:03:01.572150 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.572127    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 08 23:03:01.572150 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.572142    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 08 23:03:01.572291 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.572158    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 23:03:01.572291 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.572130    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 23:03:01.572872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.572856    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 23:03:01.572937 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.572886    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 23:03:01.572937 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.572904    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 23:03:01.573466 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.573453    2560 policy_none.go:49] "None policy: Start"
Jul 08 23:03:01.573524 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.573481    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 08 23:03:01.573524 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.573494    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 08 23:03:01.574514 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.574502    2560 policy_none.go:47] "Start"
Jul 08 23:03:01.605827 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.605811    2560 manager.go:337] "Starting Device Plugin manager"
Jul 08 23:03:01.605920 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.605850    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 08 23:03:01.605920 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.605879    2560 server.go:87] "Starting device plugin registration server"
Jul 08 23:03:01.606151 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.606137    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 08 23:03:01.606216 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.606153    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 08 23:03:01.606320 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.606299    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 08 23:03:01.606458 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.606443    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 08 23:03:01.606458 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.606457    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 08 23:03:01.606960 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.606941    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 08 23:03:01.607034 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.606989    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-54.ec2.internal\" not found"
Jul 08 23:03:01.696753 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.696709    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 08 23:03:01.698021 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.698005    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 08 23:03:01.698096 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.698045    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 08 23:03:01.698096 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.698073    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 08 23:03:01.698192 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.698112    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 08 23:03:01.700701 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.700673    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 23:03:01.706798 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.706756    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 23:03:01.707593 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.707578    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 23:03:01.707654 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.707610    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 23:03:01.707654 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.707621    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 23:03:01.707654 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.707645    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.716360 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.716345    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.716408 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.716369    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-54.ec2.internal\": node \"ip-10-0-131-54.ec2.internal\" not found"
Jul 08 23:03:01.731392 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.731371    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-54.ec2.internal\" not found"
Jul 08 23:03:01.800648 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.800604    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-54.ec2.internal"]
Jul 08 23:03:01.800748 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.800719    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 23:03:01.802560 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.802540    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 23:03:01.802643 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.802570    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 23:03:01.802643 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.802580    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 23:03:01.803995 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.803980    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 23:03:01.804129 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.804113    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.804191 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.804152    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 23:03:01.804783 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.804765    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 23:03:01.804888 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.804793    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 23:03:01.804888 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.804796    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 23:03:01.804888 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.804809    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 23:03:01.804888 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.804818    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 23:03:01.804888 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.804829    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 23:03:01.806061 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.806045    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.806148 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.806077    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 23:03:01.806777 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.806758    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 23:03:01.806863 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.806784    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 23:03:01.806863 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.806796    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeHasSufficientPID"
Jul 08 23:03:01.832262 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.832239    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-54.ec2.internal\" not found"
Jul 08 23:03:01.839417 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.839391    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-54.ec2.internal\" not found" node="ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.844358 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.844344    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-54.ec2.internal\" not found" node="ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.856037 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.856021    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e71b3d9b403cf4f726d813f56bbefbdf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal\" (UID: \"e71b3d9b403cf4f726d813f56bbefbdf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.856114 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.856046    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e71b3d9b403cf4f726d813f56bbefbdf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal\" (UID: \"e71b3d9b403cf4f726d813f56bbefbdf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.856114 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.856066    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/35a06a8130ed1bb89742a1b518abf791-config\") pod \"kube-apiserver-proxy-ip-10-0-131-54.ec2.internal\" (UID: \"35a06a8130ed1bb89742a1b518abf791\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.933548 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:01.933354    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-54.ec2.internal\" not found"
Jul 08 23:03:01.956293 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.956265    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e71b3d9b403cf4f726d813f56bbefbdf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal\" (UID: \"e71b3d9b403cf4f726d813f56bbefbdf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.956394 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.956300    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e71b3d9b403cf4f726d813f56bbefbdf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal\" (UID: \"e71b3d9b403cf4f726d813f56bbefbdf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.956394 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.956319    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/35a06a8130ed1bb89742a1b518abf791-config\") pod \"kube-apiserver-proxy-ip-10-0-131-54.ec2.internal\" (UID: \"35a06a8130ed1bb89742a1b518abf791\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.956394 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.956364    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e71b3d9b403cf4f726d813f56bbefbdf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal\" (UID: \"e71b3d9b403cf4f726d813f56bbefbdf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.956394 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.956382    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/35a06a8130ed1bb89742a1b518abf791-config\") pod \"kube-apiserver-proxy-ip-10-0-131-54.ec2.internal\" (UID: \"35a06a8130ed1bb89742a1b518abf791\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:01.956552 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:01.956364    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e71b3d9b403cf4f726d813f56bbefbdf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal\" (UID: \"e71b3d9b403cf4f726d813f56bbefbdf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:02.034645 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.034559    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-54.ec2.internal\" not found"
Jul 08 23:03:02.141094 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.141049    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-54.ec2.internal\" not found"
Jul 08 23:03:02.143388 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.143369    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:02.148891 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.148873    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:02.241694 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.241654    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-54.ec2.internal\" not found"
Jul 08 23:03:02.341942 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.341871    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-54.ec2.internal\" not found"
Jul 08 23:03:02.365478 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.365450    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 23:03:02.442610 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.442567    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-54.ec2.internal\" not found"
Jul 08 23:03:02.460682 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.460653    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 23:03:02.477764 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.477738    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 08 23:03:02.477895 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.477875    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 23:03:02.477951 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.477919    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 23:03:02.477951 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.477925    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 23:03:02.522578 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.522529    2560 apiserver.go:52] "Watching apiserver"
Jul 08 23:03:02.541890 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.541864    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 08 23:03:02.542207 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.542187    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-zmhcn","openshift-multus/multus-additional-cni-plugins-vw7ph","openshift-multus/multus-lm4nq","openshift-ovn-kubernetes/ovnkube-node-fcsjl","kube-system/global-pull-secret-syncer-w7pd9","kube-system/konnectivity-agent-r2qwv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s","openshift-dns/node-resolver-tkl7k","openshift-image-registry/node-ca-g669b","openshift-multus/network-metrics-daemon-2fss7","openshift-network-diagnostics/network-check-target-l6w79","openshift-network-operator/iptables-alerter-m6st7"]
Jul 08 23:03:02.543697 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.543667    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.545128 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.545111    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.545865 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.545846    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 08 23:03:02.546072 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.546057    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9cfj5\""
Jul 08 23:03:02.546154 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.546074    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 08 23:03:02.547152 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.547136    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.547754 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.547735    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 08 23:03:02.547820 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.547795    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lblxr\""
Jul 08 23:03:02.548289 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.548269    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 08 23:03:02.548616 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.548373    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 08 23:03:02.548768 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.548752    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 08 23:03:02.549013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.548989    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 08 23:03:02.550361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.549320    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 08 23:03:02.550361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.549569    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v4j9b\""
Jul 08 23:03:02.550519 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.550395    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.551709 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.551692    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:02.551807 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.551763    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:02.551955 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.551919    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 22:58:02 +0000 UTC" deadline="2028-04-05 07:01:25.247626124 +0000 UTC"
Jul 08 23:03:02.551955 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.551952    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15271h58m22.695676399s"
Jul 08 23:03:02.552551 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.552526    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 08 23:03:02.552620 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.552534    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 08 23:03:02.552620 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.552561    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-89jz2\""
Jul 08 23:03:02.552873 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.552855    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 08 23:03:02.552933 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.552919    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 08 23:03:02.552981 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.552929    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:02.553750 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.553728    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:02.553833 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.553767    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 08 23:03:02.553833 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.553739    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 08 23:03:02.553941 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.553767    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 08 23:03:02.554137 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.554117    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.554928 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.554909    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 08 23:03:02.555063 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.555046    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-27vjp\""
Jul 08 23:03:02.555175 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.555158    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 08 23:03:02.555653 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.555636    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tkl7k"
Jul 08 23:03:02.556102 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.556085    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vncq8\""
Jul 08 23:03:02.556442 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.556426    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 08 23:03:02.556543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.556516    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 08 23:03:02.556543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.556518    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 08 23:03:02.557067 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.557049    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g669b"
Jul 08 23:03:02.557749 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.557731    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 08 23:03:02.557913 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.557898    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 08 23:03:02.558029 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.558012    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ds8f2\""
Jul 08 23:03:02.558507 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.558491    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:02.558583 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.558545    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:02.559358 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559338    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-var-lib-kubelet\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.559446 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559370    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-tuned\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.559446 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559391    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43b89f83-671a-4942-9401-bba18aa3ff6b-tmp\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.559446 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559435    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3681f3f9-bf81-41c8-8caa-75b5123fe91b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.559638 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559447    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 08 23:03:02.559638 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559467    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-run-openvswitch\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.559638 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559511    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-systemd\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.559638 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559534    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-sys\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.559638 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559558    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 08 23:03:02.559638 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d2px\" (UniqueName: \"kubernetes.io/projected/43b89f83-671a-4942-9401-bba18aa3ff6b-kube-api-access-9d2px\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559653    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-os-release\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559683    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559683    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bf7kl\""
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-var-lib-cni-multus\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559793    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559825    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-systemd-units\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559842    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-etc-openvswitch\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559862    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-cnibin\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559877    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-etc-kubernetes\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559903    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559922    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-lib-modules\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.559934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559944    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh9s5\" (UniqueName: \"kubernetes.io/projected/903f4e45-3f42-4b97-b0e8-140303e074b5-kube-api-access-mh9s5\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.559946    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559961    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-system-cni-dir\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.559977    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-system-cni-dir\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560010    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa634889-de73-473e-915b-56355ad11639-dbus\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560046    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560064    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-socket-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560079    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62bd284c-a66b-43b4-ae40-b1b0022b355b-hosts-file\") pod \"node-resolver-tkl7k\" (UID: \"62bd284c-a66b-43b4-ae40-b1b0022b355b\") " pod="openshift-dns/node-resolver-tkl7k"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560093    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-sysctl-d\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560126    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-conf-dir\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560170    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-run\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560198    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-cnibin\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560228    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-run-k8s-cni-cncf-io\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560254    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-var-lib-kubelet\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560279    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-slash\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3681f3f9-bf81-41c8-8caa-75b5123fe91b-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.560511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560335    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr7lk\" (UniqueName: \"kubernetes.io/projected/6cb8ea5f-623b-42f4-8345-aedda4457469-kube-api-access-tr7lk\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560365    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-var-lib-openvswitch\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560397    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-log-socket\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560433    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-cni-netd\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560456    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-ovn-node-metrics-cert\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560494    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-ovnkube-script-lib\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560519    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-cni-dir\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560537    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-os-release\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560552    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cb8ea5f-623b-42f4-8345-aedda4457469-cni-binary-copy\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560568    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-socket-dir-parent\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560584    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-hostroot\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560606    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-registration-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560631    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62bd284c-a66b-43b4-ae40-b1b0022b355b-tmp-dir\") pod \"node-resolver-tkl7k\" (UID: \"62bd284c-a66b-43b4-ae40-b1b0022b355b\") " pod="openshift-dns/node-resolver-tkl7k"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560670    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwtvp\" (UniqueName: \"kubernetes.io/projected/3681f3f9-bf81-41c8-8caa-75b5123fe91b-kube-api-access-cwtvp\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560697    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-host\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560712    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3681f3f9-bf81-41c8-8caa-75b5123fe91b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.561199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560742    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-run-multus-certs\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560759    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14eb9b44-6492-4611-8769-e80480cab2d3-konnectivity-ca\") pod \"konnectivity-agent-r2qwv\" (UID: \"14eb9b44-6492-4611-8769-e80480cab2d3\") " pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560774    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-run-netns\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560789    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-run-ovn\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560811    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-ovnkube-config\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560827    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cctw6\" (UniqueName: \"kubernetes.io/projected/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-kube-api-access-cctw6\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560842    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560857    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-modprobe-d\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560882    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-sysconfig\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560915    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14eb9b44-6492-4611-8769-e80480cab2d3-agent-certs\") pod \"konnectivity-agent-r2qwv\" (UID: \"14eb9b44-6492-4611-8769-e80480cab2d3\") " pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560964    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.560997    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-run-systemd\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561024    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa634889-de73-473e-915b-56355ad11639-kubelet-config\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561057    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6vww\" (UniqueName: \"kubernetes.io/projected/62bd284c-a66b-43b4-ae40-b1b0022b355b-kube-api-access-g6vww\") pod \"node-resolver-tkl7k\" (UID: \"62bd284c-a66b-43b4-ae40-b1b0022b355b\") " pod="openshift-dns/node-resolver-tkl7k"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561084    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-kubernetes\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561107    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-node-log\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.561794 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561131    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-cni-bin\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.562256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561156    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-device-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.562256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561182    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-sysctl-conf\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.562256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.562256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561213    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m6st7"
Jul 08 23:03:02.562256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561230    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-env-overrides\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.562256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561255    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-sys-fs\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.562256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561297    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-run-netns\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.562256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561320    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-kubelet\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.562256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561346    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-var-lib-cni-bin\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.562256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.561371    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-daemon-config\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.563357 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.563339    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 08 23:03:02.563414 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.563381    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5d62l\""
Jul 08 23:03:02.563461 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.563450    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 08 23:03:02.563531 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.563465    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 23:03:02.564118 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.564103    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 08 23:03:02.570063 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.570044    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 23:03:02.570139 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.570090    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal"]
Jul 08 23:03:02.570139 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.570112    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-54.ec2.internal"
Jul 08 23:03:02.578752 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.578735    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 23:03:02.578894 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.578880    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-54.ec2.internal"]
Jul 08 23:03:02.586361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.586342    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sdqlj"
Jul 08 23:03:02.595784 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.595769    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sdqlj"
Jul 08 23:03:02.655681 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.655649    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 08 23:03:02.661638 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-sys-fs\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.661721 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661647    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-run-netns\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.661721 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661662    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-kubelet\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.661721 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661685    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-var-lib-cni-bin\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.661821 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661730    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-var-lib-cni-bin\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.661821 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661741    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-run-netns\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.661821 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661751    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-kubelet\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.661821 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661785    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-daemon-config\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.661821 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661806    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-var-lib-kubelet\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.661973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661823    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-tuned\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.661973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661812    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-sys-fs\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.661973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661839    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43b89f83-671a-4942-9401-bba18aa3ff6b-tmp\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.661973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661882    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-var-lib-kubelet\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.661973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661896    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3681f3f9-bf81-41c8-8caa-75b5123fe91b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.661973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661924    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-run-openvswitch\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.661973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661945    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-systemd\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.661973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661960    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-sys\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.661973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661975    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d2px\" (UniqueName: \"kubernetes.io/projected/43b89f83-671a-4942-9401-bba18aa3ff6b-kube-api-access-9d2px\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.661994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-os-release\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662010    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-var-lib-cni-multus\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662042    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-var-lib-cni-multus\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662040    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-systemd\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662065    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-run-openvswitch\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662094    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-os-release\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662105    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-sys\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662192    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662214    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2be52dd7-7972-4006-a2e9-45b4518167ef-host\") pod \"node-ca-g669b\" (UID: \"2be52dd7-7972-4006-a2e9-45b4518167ef\") " pod="openshift-image-registry/node-ca-g669b"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662229    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2be52dd7-7972-4006-a2e9-45b4518167ef-serviceca\") pod \"node-ca-g669b\" (UID: \"2be52dd7-7972-4006-a2e9-45b4518167ef\") " pod="openshift-image-registry/node-ca-g669b"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662245    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662260    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-systemd-units\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662275    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-etc-openvswitch\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662290    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8df2aa5-8edb-46c3-b762-a6f0feacf97c-host-slash\") pod \"iptables-alerter-m6st7\" (UID: \"d8df2aa5-8edb-46c3-b762-a6f0feacf97c\") " pod="openshift-network-operator/iptables-alerter-m6st7"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662274    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662321    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3681f3f9-bf81-41c8-8caa-75b5123fe91b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.662399 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662341    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-etc-openvswitch\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662305    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-cnibin\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662324    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-systemd-units\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662349    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-cnibin\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662366    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-etc-kubernetes\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662385    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-etc-kubernetes\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662383    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9kzt\" (UniqueName: \"kubernetes.io/projected/2b56029e-de23-4191-ac78-50d4a48ca94b-kube-api-access-q9kzt\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662386    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662409    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-lib-modules\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662426    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh9s5\" (UniqueName: \"kubernetes.io/projected/903f4e45-3f42-4b97-b0e8-140303e074b5-kube-api-access-mh9s5\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662442    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-system-cni-dir\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662456    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-system-cni-dir\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662472    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa634889-de73-473e-915b-56355ad11639-dbus\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662515    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-system-cni-dir\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662568    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-system-cni-dir\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662612    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-lib-modules\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662616    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa634889-de73-473e-915b-56355ad11639-dbus\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:02.663196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662671    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-socket-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662696    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62bd284c-a66b-43b4-ae40-b1b0022b355b-hosts-file\") pod \"node-resolver-tkl7k\" (UID: \"62bd284c-a66b-43b4-ae40-b1b0022b355b\") " pod="openshift-dns/node-resolver-tkl7k"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662722    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-sysctl-d\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.662753    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662775    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62bd284c-a66b-43b4-ae40-b1b0022b355b-hosts-file\") pod \"node-resolver-tkl7k\" (UID: \"62bd284c-a66b-43b4-ae40-b1b0022b355b\") " pod="openshift-dns/node-resolver-tkl7k"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.662816    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret podName:aa634889-de73-473e-915b-56355ad11639 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:03.162786177 +0000 UTC m=+2.092820131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret") pod "global-pull-secret-syncer-w7pd9" (UID: "aa634889-de73-473e-915b-56355ad11639") : object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662825    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-sysctl-d\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662830    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-conf-dir\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662831    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-socket-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662849    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-run\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662865    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-cnibin\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662888    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-conf-dir\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662891    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-cnibin\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662905    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-run\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662935    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-run-k8s-cni-cncf-io\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662960    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-var-lib-kubelet\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.664013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662977    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-run-k8s-cni-cncf-io\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.662984    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-slash\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663004    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-slash\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663014    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-var-lib-kubelet\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663019    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8df2aa5-8edb-46c3-b762-a6f0feacf97c-iptables-alerter-script\") pod \"iptables-alerter-m6st7\" (UID: \"d8df2aa5-8edb-46c3-b762-a6f0feacf97c\") " pod="openshift-network-operator/iptables-alerter-m6st7"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3681f3f9-bf81-41c8-8caa-75b5123fe91b-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663092    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr7lk\" (UniqueName: \"kubernetes.io/projected/6cb8ea5f-623b-42f4-8345-aedda4457469-kube-api-access-tr7lk\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-var-lib-openvswitch\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663210    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-log-socket\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663237    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-cni-netd\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663268    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-var-lib-openvswitch\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663290    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-ovn-node-metrics-cert\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663317    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-ovnkube-script-lib\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663326    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-log-socket\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663341    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-cni-netd\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663346    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld7jj\" (UniqueName: \"kubernetes.io/projected/2be52dd7-7972-4006-a2e9-45b4518167ef-kube-api-access-ld7jj\") pod \"node-ca-g669b\" (UID: \"2be52dd7-7972-4006-a2e9-45b4518167ef\") " pod="openshift-image-registry/node-ca-g669b"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663393    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-cni-dir\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.664825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663420    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-os-release\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663481    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cb8ea5f-623b-42f4-8345-aedda4457469-cni-binary-copy\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663525    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-socket-dir-parent\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663532    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3681f3f9-bf81-41c8-8caa-75b5123fe91b-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663551    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-hostroot\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663600    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-registration-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663608    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-os-release\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663626    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62bd284c-a66b-43b4-ae40-b1b0022b355b-tmp-dir\") pod \"node-resolver-tkl7k\" (UID: \"62bd284c-a66b-43b4-ae40-b1b0022b355b\") " pod="openshift-dns/node-resolver-tkl7k"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663652    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwtvp\" (UniqueName: \"kubernetes.io/projected/3681f3f9-bf81-41c8-8caa-75b5123fe91b-kube-api-access-cwtvp\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663661    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-cni-dir\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663663    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-daemon-config\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-host\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3681f3f9-bf81-41c8-8caa-75b5123fe91b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663711    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-hostroot\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663738    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-run-multus-certs\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663773    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663789    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14eb9b44-6492-4611-8769-e80480cab2d3-konnectivity-ca\") pod \"konnectivity-agent-r2qwv\" (UID: \"14eb9b44-6492-4611-8769-e80480cab2d3\") " pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663808    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-multus-socket-dir-parent\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.665649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663818    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-run-netns\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663849    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-run-ovn\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663862    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-host\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663874    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-ovnkube-config\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663901    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cctw6\" (UniqueName: \"kubernetes.io/projected/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-kube-api-access-cctw6\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663916    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-registration-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663952    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.663986    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-modprobe-d\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664010    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-sysconfig\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664034    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14eb9b44-6492-4611-8769-e80480cab2d3-agent-certs\") pod \"konnectivity-agent-r2qwv\" (UID: \"14eb9b44-6492-4611-8769-e80480cab2d3\") " pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664040    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-ovnkube-script-lib\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664082    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-run-systemd\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664111    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-298l9\" (UniqueName: \"kubernetes.io/projected/d8df2aa5-8edb-46c3-b762-a6f0feacf97c-kube-api-access-298l9\") pod \"iptables-alerter-m6st7\" (UID: \"d8df2aa5-8edb-46c3-b762-a6f0feacf97c\") " pod="openshift-network-operator/iptables-alerter-m6st7"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664143    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cb8ea5f-623b-42f4-8345-aedda4457469-cni-binary-copy\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664176    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664205    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa634889-de73-473e-915b-56355ad11639-kubelet-config\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:02.666577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664231    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6vww\" (UniqueName: \"kubernetes.io/projected/62bd284c-a66b-43b4-ae40-b1b0022b355b-kube-api-access-g6vww\") pod \"node-resolver-tkl7k\" (UID: \"62bd284c-a66b-43b4-ae40-b1b0022b355b\") " pod="openshift-dns/node-resolver-tkl7k"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664261    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-kubernetes\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664286    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-node-log\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664306    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62bd284c-a66b-43b4-ae40-b1b0022b355b-tmp-dir\") pod \"node-resolver-tkl7k\" (UID: \"62bd284c-a66b-43b4-ae40-b1b0022b355b\") " pod="openshift-dns/node-resolver-tkl7k"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664363    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-cni-bin\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664393    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-device-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664419    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-sysctl-conf\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664446    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664472    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-env-overrides\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664500    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86pl5\" (UniqueName: \"kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5\") pod \"network-check-target-l6w79\" (UID: \"6952cf89-883f-4132-8357-52b4f055486f\") " pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664613    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3681f3f9-bf81-41c8-8caa-75b5123fe91b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664631    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-run-netns\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664681    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-run-ovn\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664694    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14eb9b44-6492-4611-8769-e80480cab2d3-konnectivity-ca\") pod \"konnectivity-agent-r2qwv\" (UID: \"14eb9b44-6492-4611-8769-e80480cab2d3\") " pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664681    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6cb8ea5f-623b-42f4-8345-aedda4457469-host-run-multus-certs\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664752    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-cni-bin\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664786    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-modprobe-d\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.667373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664808    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa634889-de73-473e-915b-56355ad11639-kubelet-config\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664833    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-sysconfig\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664878    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-device-dir\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664909    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-sysctl-conf\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.664929    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-kubernetes\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.665034    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-node-log\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.665074    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3681f3f9-bf81-41c8-8caa-75b5123fe91b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.665095    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-run-systemd\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.665130    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.665228    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/903f4e45-3f42-4b97-b0e8-140303e074b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.665299    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-env-overrides\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.665470    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-ovnkube-config\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.667395    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43b89f83-671a-4942-9401-bba18aa3ff6b-etc-tuned\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.667452    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43b89f83-671a-4942-9401-bba18aa3ff6b-tmp\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.667515    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-ovn-node-metrics-cert\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.667967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.667567    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14eb9b44-6492-4611-8769-e80480cab2d3-agent-certs\") pod \"konnectivity-agent-r2qwv\" (UID: \"14eb9b44-6492-4611-8769-e80480cab2d3\") " pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:02.672855 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.672833    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d2px\" (UniqueName: \"kubernetes.io/projected/43b89f83-671a-4942-9401-bba18aa3ff6b-kube-api-access-9d2px\") pod \"tuned-zmhcn\" (UID: \"43b89f83-671a-4942-9401-bba18aa3ff6b\") " pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.676981 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.676954    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh9s5\" (UniqueName: \"kubernetes.io/projected/903f4e45-3f42-4b97-b0e8-140303e074b5-kube-api-access-mh9s5\") pod \"aws-ebs-csi-driver-node-9kf8s\" (UID: \"903f4e45-3f42-4b97-b0e8-140303e074b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.677260 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.677239    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s"
Jul 08 23:03:02.677768 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.677744    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr7lk\" (UniqueName: \"kubernetes.io/projected/6cb8ea5f-623b-42f4-8345-aedda4457469-kube-api-access-tr7lk\") pod \"multus-lm4nq\" (UID: \"6cb8ea5f-623b-42f4-8345-aedda4457469\") " pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.677973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.677952    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwtvp\" (UniqueName: \"kubernetes.io/projected/3681f3f9-bf81-41c8-8caa-75b5123fe91b-kube-api-access-cwtvp\") pod \"multus-additional-cni-plugins-vw7ph\" (UID: \"3681f3f9-bf81-41c8-8caa-75b5123fe91b\") " pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.678046 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.677954    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6vww\" (UniqueName: \"kubernetes.io/projected/62bd284c-a66b-43b4-ae40-b1b0022b355b-kube-api-access-g6vww\") pod \"node-resolver-tkl7k\" (UID: \"62bd284c-a66b-43b4-ae40-b1b0022b355b\") " pod="openshift-dns/node-resolver-tkl7k"
Jul 08 23:03:02.678615 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.678596    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cctw6\" (UniqueName: \"kubernetes.io/projected/221668dd-86f4-4e20-bc7f-12cd2d4ce3d6-kube-api-access-cctw6\") pod \"ovnkube-node-fcsjl\" (UID: \"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.702258 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.702238    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tkl7k"
Jul 08 23:03:02.764736 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.764694    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9kzt\" (UniqueName: \"kubernetes.io/projected/2b56029e-de23-4191-ac78-50d4a48ca94b-kube-api-access-q9kzt\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:02.765025 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.764765    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8df2aa5-8edb-46c3-b762-a6f0feacf97c-iptables-alerter-script\") pod \"iptables-alerter-m6st7\" (UID: \"d8df2aa5-8edb-46c3-b762-a6f0feacf97c\") " pod="openshift-network-operator/iptables-alerter-m6st7"
Jul 08 23:03:02.765025 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.764787    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld7jj\" (UniqueName: \"kubernetes.io/projected/2be52dd7-7972-4006-a2e9-45b4518167ef-kube-api-access-ld7jj\") pod \"node-ca-g669b\" (UID: \"2be52dd7-7972-4006-a2e9-45b4518167ef\") " pod="openshift-image-registry/node-ca-g669b"
Jul 08 23:03:02.765025 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.764827    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-298l9\" (UniqueName: \"kubernetes.io/projected/d8df2aa5-8edb-46c3-b762-a6f0feacf97c-kube-api-access-298l9\") pod \"iptables-alerter-m6st7\" (UID: \"d8df2aa5-8edb-46c3-b762-a6f0feacf97c\") " pod="openshift-network-operator/iptables-alerter-m6st7"
Jul 08 23:03:02.765025 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.764843    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:02.765025 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.764881    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86pl5\" (UniqueName: \"kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5\") pod \"network-check-target-l6w79\" (UID: \"6952cf89-883f-4132-8357-52b4f055486f\") " pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:02.765317 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.765203    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2be52dd7-7972-4006-a2e9-45b4518167ef-host\") pod \"node-ca-g669b\" (UID: \"2be52dd7-7972-4006-a2e9-45b4518167ef\") " pod="openshift-image-registry/node-ca-g669b"
Jul 08 23:03:02.765317 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.765236    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:02.765317 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.765238    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2be52dd7-7972-4006-a2e9-45b4518167ef-serviceca\") pod \"node-ca-g669b\" (UID: \"2be52dd7-7972-4006-a2e9-45b4518167ef\") " pod="openshift-image-registry/node-ca-g669b"
Jul 08 23:03:02.765317 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.765288    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2be52dd7-7972-4006-a2e9-45b4518167ef-host\") pod \"node-ca-g669b\" (UID: \"2be52dd7-7972-4006-a2e9-45b4518167ef\") " pod="openshift-image-registry/node-ca-g669b"
Jul 08 23:03:02.765317 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.765312    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs podName:2b56029e-de23-4191-ac78-50d4a48ca94b nodeName:}" failed. No retries permitted until 2026-07-08 23:03:03.265290758 +0000 UTC m=+2.195324730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs") pod "network-metrics-daemon-2fss7" (UID: "2b56029e-de23-4191-ac78-50d4a48ca94b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:02.765588 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.765355    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8df2aa5-8edb-46c3-b762-a6f0feacf97c-host-slash\") pod \"iptables-alerter-m6st7\" (UID: \"d8df2aa5-8edb-46c3-b762-a6f0feacf97c\") " pod="openshift-network-operator/iptables-alerter-m6st7"
Jul 08 23:03:02.765588 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.765417    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8df2aa5-8edb-46c3-b762-a6f0feacf97c-host-slash\") pod \"iptables-alerter-m6st7\" (UID: \"d8df2aa5-8edb-46c3-b762-a6f0feacf97c\") " pod="openshift-network-operator/iptables-alerter-m6st7"
Jul 08 23:03:02.765588 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.765502    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d8df2aa5-8edb-46c3-b762-a6f0feacf97c-iptables-alerter-script\") pod \"iptables-alerter-m6st7\" (UID: \"d8df2aa5-8edb-46c3-b762-a6f0feacf97c\") " pod="openshift-network-operator/iptables-alerter-m6st7"
Jul 08 23:03:02.766291 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.766272    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2be52dd7-7972-4006-a2e9-45b4518167ef-serviceca\") pod \"node-ca-g669b\" (UID: \"2be52dd7-7972-4006-a2e9-45b4518167ef\") " pod="openshift-image-registry/node-ca-g669b"
Jul 08 23:03:02.772130 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.772095    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 23:03:02.772130 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.772125    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 23:03:02.772315 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.772139    2560 projected.go:196] Error preparing data for projected volume kube-api-access-86pl5 for pod openshift-network-diagnostics/network-check-target-l6w79: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:02.772315 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:02.772210    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5 podName:6952cf89-883f-4132-8357-52b4f055486f nodeName:}" failed. No retries permitted until 2026-07-08 23:03:03.272190816 +0000 UTC m=+2.202224786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-86pl5" (UniqueName: "kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5") pod "network-check-target-l6w79" (UID: "6952cf89-883f-4132-8357-52b4f055486f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:02.774019 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.773992    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9kzt\" (UniqueName: \"kubernetes.io/projected/2b56029e-de23-4191-ac78-50d4a48ca94b-kube-api-access-q9kzt\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:02.774119 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.773988    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld7jj\" (UniqueName: \"kubernetes.io/projected/2be52dd7-7972-4006-a2e9-45b4518167ef-kube-api-access-ld7jj\") pod \"node-ca-g669b\" (UID: \"2be52dd7-7972-4006-a2e9-45b4518167ef\") " pod="openshift-image-registry/node-ca-g669b"
Jul 08 23:03:02.774604 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.774581    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-298l9\" (UniqueName: \"kubernetes.io/projected/d8df2aa5-8edb-46c3-b762-a6f0feacf97c-kube-api-access-298l9\") pod \"iptables-alerter-m6st7\" (UID: \"d8df2aa5-8edb-46c3-b762-a6f0feacf97c\") " pod="openshift-network-operator/iptables-alerter-m6st7"
Jul 08 23:03:02.870782 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.870708    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zmhcn"
Jul 08 23:03:02.889788 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:02.889756    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode71b3d9b403cf4f726d813f56bbefbdf.slice/crio-66656905d7f4b4b1edf5e636d8c267d26f4655eb12f170b59a62cb9ddf1ddcc4 WatchSource:0}: Error finding container 66656905d7f4b4b1edf5e636d8c267d26f4655eb12f170b59a62cb9ddf1ddcc4: Status 404 returned error can't find the container with id 66656905d7f4b4b1edf5e636d8c267d26f4655eb12f170b59a62cb9ddf1ddcc4
Jul 08 23:03:02.892113 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:02.892071    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b89f83_671a_4942_9401_bba18aa3ff6b.slice/crio-b55084a48099ed29a6e638e1726d29ab8ee519ed50361673a4f9ce9d3db52a6a WatchSource:0}: Error finding container b55084a48099ed29a6e638e1726d29ab8ee519ed50361673a4f9ce9d3db52a6a: Status 404 returned error can't find the container with id b55084a48099ed29a6e638e1726d29ab8ee519ed50361673a4f9ce9d3db52a6a
Jul 08 23:03:02.894431 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.894410    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vw7ph"
Jul 08 23:03:02.898884 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.898864    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 23:03:02.903224 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:02.903194    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35a06a8130ed1bb89742a1b518abf791.slice/crio-a5b076a93bd404dc043463b6a197fa0d28dd6c0dab1ec7003682cca96d4c0b8d WatchSource:0}: Error finding container a5b076a93bd404dc043463b6a197fa0d28dd6c0dab1ec7003682cca96d4c0b8d: Status 404 returned error can't find the container with id a5b076a93bd404dc043463b6a197fa0d28dd6c0dab1ec7003682cca96d4c0b8d
Jul 08 23:03:02.905137 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:02.905064    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3681f3f9_bf81_41c8_8caa_75b5123fe91b.slice/crio-b8aa2d12171b2cb646eb75f21db0b46cef267aaad8e31a600f42829865490f3a WatchSource:0}: Error finding container b8aa2d12171b2cb646eb75f21db0b46cef267aaad8e31a600f42829865490f3a: Status 404 returned error can't find the container with id b8aa2d12171b2cb646eb75f21db0b46cef267aaad8e31a600f42829865490f3a
Jul 08 23:03:02.920730 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.920706    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lm4nq"
Jul 08 23:03:02.931053 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:02.931027    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb8ea5f_623b_42f4_8345_aedda4457469.slice/crio-40df12b0f70d9bb2a16c21c7ef1bccd2fd30a6a796498ba39e3a6ccb0e2467e1 WatchSource:0}: Error finding container 40df12b0f70d9bb2a16c21c7ef1bccd2fd30a6a796498ba39e3a6ccb0e2467e1: Status 404 returned error can't find the container with id 40df12b0f70d9bb2a16c21c7ef1bccd2fd30a6a796498ba39e3a6ccb0e2467e1
Jul 08 23:03:02.935425 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.935408    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:02.943942 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:02.943919    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod221668dd_86f4_4e20_bc7f_12cd2d4ce3d6.slice/crio-b87d88b16bbf6ac735d9fb492dee0493a3e5787e0952ef9f0684363015bf02e2 WatchSource:0}: Error finding container b87d88b16bbf6ac735d9fb492dee0493a3e5787e0952ef9f0684363015bf02e2: Status 404 returned error can't find the container with id b87d88b16bbf6ac735d9fb492dee0493a3e5787e0952ef9f0684363015bf02e2
Jul 08 23:03:02.956502 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.956482    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:02.965005 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:02.964982    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14eb9b44_6492_4611_8769_e80480cab2d3.slice/crio-b876b0e18eb361e9535d8cd7699827e87d9bfaedbeb5d0fe8892206202dae2c7 WatchSource:0}: Error finding container b876b0e18eb361e9535d8cd7699827e87d9bfaedbeb5d0fe8892206202dae2c7: Status 404 returned error can't find the container with id b876b0e18eb361e9535d8cd7699827e87d9bfaedbeb5d0fe8892206202dae2c7
Jul 08 23:03:02.972305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:02.972285    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 23:03:03.018155 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.018119    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g669b"
Jul 08 23:03:03.023560 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.023542    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m6st7"
Jul 08 23:03:03.027380 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:03.027351    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be52dd7_7972_4006_a2e9_45b4518167ef.slice/crio-9f34fa7acc88063cf470898518216fe53fc0b6c3235c7e99b1b33f1a0773d92b WatchSource:0}: Error finding container 9f34fa7acc88063cf470898518216fe53fc0b6c3235c7e99b1b33f1a0773d92b: Status 404 returned error can't find the container with id 9f34fa7acc88063cf470898518216fe53fc0b6c3235c7e99b1b33f1a0773d92b
Jul 08 23:03:03.033186 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:03.033158    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8df2aa5_8edb_46c3_b762_a6f0feacf97c.slice/crio-094c3ff44267b8f3e5e5602fee52b6e00b2fcbaf1f2a8c7347d9d8b65e6aec85 WatchSource:0}: Error finding container 094c3ff44267b8f3e5e5602fee52b6e00b2fcbaf1f2a8c7347d9d8b65e6aec85: Status 404 returned error can't find the container with id 094c3ff44267b8f3e5e5602fee52b6e00b2fcbaf1f2a8c7347d9d8b65e6aec85
Jul 08 23:03:03.087565 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:03.087527    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod903f4e45_3f42_4b97_b0e8_140303e074b5.slice/crio-acf8176c65bc1ab610c6d47fbab7d21e9c995aaa9f32407e155f18ba6108229a WatchSource:0}: Error finding container acf8176c65bc1ab610c6d47fbab7d21e9c995aaa9f32407e155f18ba6108229a: Status 404 returned error can't find the container with id acf8176c65bc1ab610c6d47fbab7d21e9c995aaa9f32407e155f18ba6108229a
Jul 08 23:03:03.139497 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:03.139465    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62bd284c_a66b_43b4_ae40_b1b0022b355b.slice/crio-b80531bc566d3a5a936bd3cfdb310f8b56fa22dc8abcd67443fa85b59211c054 WatchSource:0}: Error finding container b80531bc566d3a5a936bd3cfdb310f8b56fa22dc8abcd67443fa85b59211c054: Status 404 returned error can't find the container with id b80531bc566d3a5a936bd3cfdb310f8b56fa22dc8abcd67443fa85b59211c054
Jul 08 23:03:03.167680 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.167642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:03.167816 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:03.167784    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:03.167857 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:03.167844    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret podName:aa634889-de73-473e-915b-56355ad11639 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:04.167829793 +0000 UTC m=+3.097863745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret") pod "global-pull-secret-syncer-w7pd9" (UID: "aa634889-de73-473e-915b-56355ad11639") : object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:03.268429 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.268398    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:03.268612 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:03.268568    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:03.268654 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:03.268630    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs podName:2b56029e-de23-4191-ac78-50d4a48ca94b nodeName:}" failed. No retries permitted until 2026-07-08 23:03:04.268613878 +0000 UTC m=+3.198647833 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs") pod "network-metrics-daemon-2fss7" (UID: "2b56029e-de23-4191-ac78-50d4a48ca94b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:03.369157 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.369116    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86pl5\" (UniqueName: \"kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5\") pod \"network-check-target-l6w79\" (UID: \"6952cf89-883f-4132-8357-52b4f055486f\") " pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:03.369339 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:03.369293    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 23:03:03.369339 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:03.369313    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 23:03:03.369339 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:03.369328    2560 projected.go:196] Error preparing data for projected volume kube-api-access-86pl5 for pod openshift-network-diagnostics/network-check-target-l6w79: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:03.369535 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:03.369385    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5 podName:6952cf89-883f-4132-8357-52b4f055486f nodeName:}" failed. No retries permitted until 2026-07-08 23:03:04.369365861 +0000 UTC m=+3.299399820 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-86pl5" (UniqueName: "kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5") pod "network-check-target-l6w79" (UID: "6952cf89-883f-4132-8357-52b4f055486f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:03.598153 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.598031    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 22:58:03 +0000 UTC" deadline="2028-04-04 03:29:40.754640042 +0000 UTC"
Jul 08 23:03:03.598153 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.598094    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15244h26m37.156550769s"
Jul 08 23:03:03.636130 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.636066    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 23:03:03.702413 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.701732    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:03.702413 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:03.701865    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:03.730725 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.730662    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tkl7k" event={"ID":"62bd284c-a66b-43b4-ae40-b1b0022b355b","Type":"ContainerStarted","Data":"b80531bc566d3a5a936bd3cfdb310f8b56fa22dc8abcd67443fa85b59211c054"}
Jul 08 23:03:03.749632 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.749594    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g669b" event={"ID":"2be52dd7-7972-4006-a2e9-45b4518167ef","Type":"ContainerStarted","Data":"9f34fa7acc88063cf470898518216fe53fc0b6c3235c7e99b1b33f1a0773d92b"}
Jul 08 23:03:03.752617 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.752585    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r2qwv" event={"ID":"14eb9b44-6492-4611-8769-e80480cab2d3","Type":"ContainerStarted","Data":"b876b0e18eb361e9535d8cd7699827e87d9bfaedbeb5d0fe8892206202dae2c7"}
Jul 08 23:03:03.772539 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.772501    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw7ph" event={"ID":"3681f3f9-bf81-41c8-8caa-75b5123fe91b","Type":"ContainerStarted","Data":"b8aa2d12171b2cb646eb75f21db0b46cef267aaad8e31a600f42829865490f3a"}
Jul 08 23:03:03.786003 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.785967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal" event={"ID":"e71b3d9b403cf4f726d813f56bbefbdf","Type":"ContainerStarted","Data":"66656905d7f4b4b1edf5e636d8c267d26f4655eb12f170b59a62cb9ddf1ddcc4"}
Jul 08 23:03:03.831220 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.831174    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s" event={"ID":"903f4e45-3f42-4b97-b0e8-140303e074b5","Type":"ContainerStarted","Data":"acf8176c65bc1ab610c6d47fbab7d21e9c995aaa9f32407e155f18ba6108229a"}
Jul 08 23:03:03.860396 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.860295    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m6st7" event={"ID":"d8df2aa5-8edb-46c3-b762-a6f0feacf97c","Type":"ContainerStarted","Data":"094c3ff44267b8f3e5e5602fee52b6e00b2fcbaf1f2a8c7347d9d8b65e6aec85"}
Jul 08 23:03:03.869937 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.869880    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl" event={"ID":"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6","Type":"ContainerStarted","Data":"b87d88b16bbf6ac735d9fb492dee0493a3e5787e0952ef9f0684363015bf02e2"}
Jul 08 23:03:03.897736 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.897627    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lm4nq" event={"ID":"6cb8ea5f-623b-42f4-8345-aedda4457469","Type":"ContainerStarted","Data":"40df12b0f70d9bb2a16c21c7ef1bccd2fd30a6a796498ba39e3a6ccb0e2467e1"}
Jul 08 23:03:03.909203 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.909160    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-54.ec2.internal" event={"ID":"35a06a8130ed1bb89742a1b518abf791","Type":"ContainerStarted","Data":"a5b076a93bd404dc043463b6a197fa0d28dd6c0dab1ec7003682cca96d4c0b8d"}
Jul 08 23:03:03.932638 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:03.932602    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zmhcn" event={"ID":"43b89f83-671a-4942-9401-bba18aa3ff6b","Type":"ContainerStarted","Data":"b55084a48099ed29a6e638e1726d29ab8ee519ed50361673a4f9ce9d3db52a6a"}
Jul 08 23:03:04.177694 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:04.176756    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:04.177694 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:04.176960    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:04.177694 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:04.177026    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret podName:aa634889-de73-473e-915b-56355ad11639 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:06.177007189 +0000 UTC m=+5.107041145 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret") pod "global-pull-secret-syncer-w7pd9" (UID: "aa634889-de73-473e-915b-56355ad11639") : object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:04.278149 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:04.278107    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:04.278348 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:04.278312    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:04.278418 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:04.278381    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs podName:2b56029e-de23-4191-ac78-50d4a48ca94b nodeName:}" failed. No retries permitted until 2026-07-08 23:03:06.278359093 +0000 UTC m=+5.208393048 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs") pod "network-metrics-daemon-2fss7" (UID: "2b56029e-de23-4191-ac78-50d4a48ca94b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:04.374112 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:04.374076    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 23:03:04.379386 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:04.379352    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86pl5\" (UniqueName: \"kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5\") pod \"network-check-target-l6w79\" (UID: \"6952cf89-883f-4132-8357-52b4f055486f\") " pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:04.379609 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:04.379591    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 23:03:04.379687 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:04.379618    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 23:03:04.379687 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:04.379632    2560 projected.go:196] Error preparing data for projected volume kube-api-access-86pl5 for pod openshift-network-diagnostics/network-check-target-l6w79: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:04.379805 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:04.379691    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5 podName:6952cf89-883f-4132-8357-52b4f055486f nodeName:}" failed. No retries permitted until 2026-07-08 23:03:06.379671458 +0000 UTC m=+5.309705435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-86pl5" (UniqueName: "kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5") pod "network-check-target-l6w79" (UID: "6952cf89-883f-4132-8357-52b4f055486f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:04.598825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:04.598712    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 22:58:03 +0000 UTC" deadline="2028-03-06 00:23:03.71072415 +0000 UTC"
Jul 08 23:03:04.598825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:04.598754    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14545h19m59.111975455s"
Jul 08 23:03:04.694633 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:04.694601    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 23:03:04.698825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:04.698798    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:04.698985 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:04.698945    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:04.699423 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:04.699400    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:04.699533 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:04.699506    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:05.704674 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:05.704641    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:05.705296 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:05.704794    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:06.196113 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:06.196064    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:06.196297 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:06.196282    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:06.196374 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:06.196348    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret podName:aa634889-de73-473e-915b-56355ad11639 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:10.196325872 +0000 UTC m=+9.126359833 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret") pod "global-pull-secret-syncer-w7pd9" (UID: "aa634889-de73-473e-915b-56355ad11639") : object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:06.297678 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:06.297641    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:06.297868 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:06.297799    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:06.297868 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:06.297864    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs podName:2b56029e-de23-4191-ac78-50d4a48ca94b nodeName:}" failed. No retries permitted until 2026-07-08 23:03:10.297843738 +0000 UTC m=+9.227877707 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs") pod "network-metrics-daemon-2fss7" (UID: "2b56029e-de23-4191-ac78-50d4a48ca94b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:06.398671 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:06.398598    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86pl5\" (UniqueName: \"kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5\") pod \"network-check-target-l6w79\" (UID: \"6952cf89-883f-4132-8357-52b4f055486f\") " pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:06.398874 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:06.398851    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 23:03:06.398949 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:06.398876    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 23:03:06.398949 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:06.398890    2560 projected.go:196] Error preparing data for projected volume kube-api-access-86pl5 for pod openshift-network-diagnostics/network-check-target-l6w79: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:06.399062 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:06.398952    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5 podName:6952cf89-883f-4132-8357-52b4f055486f nodeName:}" failed. No retries permitted until 2026-07-08 23:03:10.398931867 +0000 UTC m=+9.328965827 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-86pl5" (UniqueName: "kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5") pod "network-check-target-l6w79" (UID: "6952cf89-883f-4132-8357-52b4f055486f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:06.698950 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:06.698917    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:06.699127 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:06.699064    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:06.699211 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:06.699123    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:06.699338 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:06.699255    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:07.699992 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:07.699449    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:07.699992 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:07.699607    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:08.698768 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:08.698732    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:08.698965 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:08.698732    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:08.698965 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:08.698926    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:08.699090 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:08.699068    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:09.699176 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:09.699141    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:09.699674 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:09.699285    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:10.235352 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:10.235311    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:10.235560 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:10.235520    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:10.235632 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:10.235601    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret podName:aa634889-de73-473e-915b-56355ad11639 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:18.235577129 +0000 UTC m=+17.165611157 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret") pod "global-pull-secret-syncer-w7pd9" (UID: "aa634889-de73-473e-915b-56355ad11639") : object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:10.336570 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:10.336453    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:10.336777 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:10.336647    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:10.336777 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:10.336735    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs podName:2b56029e-de23-4191-ac78-50d4a48ca94b nodeName:}" failed. No retries permitted until 2026-07-08 23:03:18.336711953 +0000 UTC m=+17.266745911 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs") pod "network-metrics-daemon-2fss7" (UID: "2b56029e-de23-4191-ac78-50d4a48ca94b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:10.438404 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:10.437710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86pl5\" (UniqueName: \"kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5\") pod \"network-check-target-l6w79\" (UID: \"6952cf89-883f-4132-8357-52b4f055486f\") " pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:10.438404 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:10.437910    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 23:03:10.438404 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:10.437933    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 23:03:10.438404 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:10.437947    2560 projected.go:196] Error preparing data for projected volume kube-api-access-86pl5 for pod openshift-network-diagnostics/network-check-target-l6w79: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:10.438404 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:10.438009    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5 podName:6952cf89-883f-4132-8357-52b4f055486f nodeName:}" failed. No retries permitted until 2026-07-08 23:03:18.437989911 +0000 UTC m=+17.368023869 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-86pl5" (UniqueName: "kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5") pod "network-check-target-l6w79" (UID: "6952cf89-883f-4132-8357-52b4f055486f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:10.699091 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:10.698711    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:10.699091 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:10.698739    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:10.699091 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:10.698856    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:10.699091 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:10.698972    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:11.699963 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:11.699924    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:11.700548 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:11.700040    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:12.698632 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:12.698600    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:12.698809 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:12.698643    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:12.698809 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:12.698728    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:12.698926 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:12.698864    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:13.699256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:13.699226    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:13.699696 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:13.699338    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:14.699270 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:14.699243    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:14.699646 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:14.699331    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:14.699646 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:14.699393    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:14.699646 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:14.699491    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:15.698584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:15.698550    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:15.698752 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:15.698680    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:16.698661 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:16.698627    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:16.699087 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:16.698743    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:16.699087 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:16.698796    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:16.699087 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:16.698950    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:17.698780 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:17.698741    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:17.699242 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:17.698896    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:18.298608 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:18.298570    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:18.298786 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:18.298721    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:18.298832 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:18.298799    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret podName:aa634889-de73-473e-915b-56355ad11639 nodeName:}" failed. No retries permitted until 2026-07-08 23:03:34.298779866 +0000 UTC m=+33.228813818 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret") pod "global-pull-secret-syncer-w7pd9" (UID: "aa634889-de73-473e-915b-56355ad11639") : object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:18.399929 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:18.399891    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:18.400093 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:18.400009    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:18.400093 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:18.400064    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs podName:2b56029e-de23-4191-ac78-50d4a48ca94b nodeName:}" failed. No retries permitted until 2026-07-08 23:03:34.400048763 +0000 UTC m=+33.330082715 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs") pod "network-metrics-daemon-2fss7" (UID: "2b56029e-de23-4191-ac78-50d4a48ca94b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:18.500181 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:18.500143    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86pl5\" (UniqueName: \"kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5\") pod \"network-check-target-l6w79\" (UID: \"6952cf89-883f-4132-8357-52b4f055486f\") " pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:18.500345 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:18.500263    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 23:03:18.500345 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:18.500285    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 23:03:18.500345 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:18.500296    2560 projected.go:196] Error preparing data for projected volume kube-api-access-86pl5 for pod openshift-network-diagnostics/network-check-target-l6w79: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:18.500509 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:18.500348    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5 podName:6952cf89-883f-4132-8357-52b4f055486f nodeName:}" failed. No retries permitted until 2026-07-08 23:03:34.500334391 +0000 UTC m=+33.430368344 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-86pl5" (UniqueName: "kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5") pod "network-check-target-l6w79" (UID: "6952cf89-883f-4132-8357-52b4f055486f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:18.698419 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:18.698335    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:18.698578 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:18.698335    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:18.698578 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:18.698458    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:18.698686 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:18.698588    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:19.698659 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:19.698628    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:19.699118 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:19.698750    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:20.698607 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:20.698573    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:20.698804 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:20.698620    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:20.698804 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:20.698714    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:20.699223 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:20.698953    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:21.699581 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:21.699544    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:21.700041 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:21.699728    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:22.699141 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:22.699101    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:22.699356 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:22.699152    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:22.699356 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:22.699230    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:22.699496 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:22.699384    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:23.698449 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:23.698418    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:23.698910 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:23.698560    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:23.996276 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:23.996091    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lm4nq" event={"ID":"6cb8ea5f-623b-42f4-8345-aedda4457469","Type":"ContainerStarted","Data":"85abf3aed60182a5a927da94b8281c9aa1942847e8ab393d4f78b283f287516b"}
Jul 08 23:03:23.997578 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:23.997553    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-54.ec2.internal" event={"ID":"35a06a8130ed1bb89742a1b518abf791","Type":"ContainerStarted","Data":"8339d875eeb3ae2cb66967edb4ca999d93a93107b24af6b6a49000a69bd9bc3b"}
Jul 08 23:03:23.998821 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:23.998799    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zmhcn" event={"ID":"43b89f83-671a-4942-9401-bba18aa3ff6b","Type":"ContainerStarted","Data":"a770391810f746ef42c78d7d0d6ca579e806962e0d02c3c9f47d047f9d4d89c4"}
Jul 08 23:03:24.012275 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:24.012235    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lm4nq" podStartSLOduration=1.2056521980000001 podStartE2EDuration="22.012222698s" totalImagesPullingTime="20.8065705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:24.012170568 +0000 UTC m=+22.942204543" watchObservedRunningTime="2026-07-08 23:03:24.012222698 +0000 UTC m=+22.942256688"
Jul 08 23:03:24.024632 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:24.024594    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-54.ec2.internal" podStartSLOduration=21.024579657 podStartE2EDuration="21.024579657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:24.024177753 +0000 UTC m=+22.954211724" watchObservedRunningTime="2026-07-08 23:03:24.024579657 +0000 UTC m=+22.954613631"
Jul 08 23:03:24.038310 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:24.038272    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zmhcn" podStartSLOduration=1.214115937 podStartE2EDuration="22.038259003s" totalImagesPullingTime="20.824143066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:24.038042382 +0000 UTC m=+22.968076359" watchObservedRunningTime="2026-07-08 23:03:24.038259003 +0000 UTC m=+22.968292977"
Jul 08 23:03:24.698682 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:24.698482    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:24.699245 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:24.698545    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:24.699245 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:24.698783    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:24.699245 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:24.698855    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:25.003178 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:25.003106    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:03:25.003444 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:25.003422    2560 generic.go:356] "Generic (PLEG): container finished" podID="221668dd-86f4-4e20-bc7f-12cd2d4ce3d6" containerID="31a3a2d754d80299d8192da2487ba59bf48e1452ca5380875ed8ef7c7a3e3ced" exitCode=1
Jul 08 23:03:25.003565 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:25.003522    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl" event={"ID":"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6","Type":"ContainerStarted","Data":"92c9d76ac6e331015ff436b37be465aa3c927acb5232f47f4f8d11fc0d6bcbab"}
Jul 08 23:03:25.003632 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:25.003590    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl" event={"ID":"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6","Type":"ContainerStarted","Data":"6c2d9645e37441c239f79c81740274f7fe742329545c5f62db2f00e8953de03c"}
Jul 08 23:03:25.003673 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:25.003636    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl" event={"ID":"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6","Type":"ContainerStarted","Data":"df5d6444da4ffb6ec55eefcf2a6924fe9d22568efb9db895d9299bda37774271"}
Jul 08 23:03:25.003713 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:25.003685    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl" event={"ID":"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6","Type":"ContainerStarted","Data":"e7eef8b63c5b2c8c0d0b4a03d49da755c0931a9e630592b28c452c4d44affebd"}
Jul 08 23:03:25.003754 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:25.003732    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl" event={"ID":"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6","Type":"ContainerDied","Data":"31a3a2d754d80299d8192da2487ba59bf48e1452ca5380875ed8ef7c7a3e3ced"}
Jul 08 23:03:25.003786 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:25.003778    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl" event={"ID":"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6","Type":"ContainerStarted","Data":"9a66d0e28eb74bb77e7c2f9d0c5de353ae6a4f15d4388c068c39606126f37de4"}
Jul 08 23:03:25.698443 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:25.698417    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:25.698613 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:25.698535    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:26.008247 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.008159    2560 generic.go:356] "Generic (PLEG): container finished" podID="e71b3d9b403cf4f726d813f56bbefbdf" containerID="7fa681ffb2274c10c15b15ff9457e3c1c88760910899b479a80636daba4e7617" exitCode=0
Jul 08 23:03:26.009191 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.008243    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal" event={"ID":"e71b3d9b403cf4f726d813f56bbefbdf","Type":"ContainerDied","Data":"7fa681ffb2274c10c15b15ff9457e3c1c88760910899b479a80636daba4e7617"}
Jul 08 23:03:26.009849 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.009815    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s" event={"ID":"903f4e45-3f42-4b97-b0e8-140303e074b5","Type":"ContainerStarted","Data":"1d9e40a9c5c32cd8e2e1437ae682650b5de648486f2f1bd93c8cdf2f2762c023"}
Jul 08 23:03:26.011513 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.011487    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m6st7" event={"ID":"d8df2aa5-8edb-46c3-b762-a6f0feacf97c","Type":"ContainerStarted","Data":"0aa1aba726352a15442cad913706938bd43980eb7487c1aaec5d411d6e7644c8"}
Jul 08 23:03:26.013415 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.013388    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tkl7k" event={"ID":"62bd284c-a66b-43b4-ae40-b1b0022b355b","Type":"ContainerStarted","Data":"02cdf4d96624d769a1cd7b6fbfaa2e24b0f56bd1828e3eedf33d9ef185f28a38"}
Jul 08 23:03:26.015316 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.015287    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g669b" event={"ID":"2be52dd7-7972-4006-a2e9-45b4518167ef","Type":"ContainerStarted","Data":"60298a26d38843474050bbbd0e7c205d24cb170401a2f01e850c37e82f79f7c7"}
Jul 08 23:03:26.018729 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.018706    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r2qwv" event={"ID":"14eb9b44-6492-4611-8769-e80480cab2d3","Type":"ContainerStarted","Data":"2bcc11eb615ab9b2ae8c38cfc82245740a01fd556c46d67abeaab89bc134e7e5"}
Jul 08 23:03:26.020144 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.020119    2560 generic.go:356] "Generic (PLEG): container finished" podID="3681f3f9-bf81-41c8-8caa-75b5123fe91b" containerID="0acf0255e8cf31477ac0923406de6327118debb049dcb5ce7a7a3e94a60685a8" exitCode=0
Jul 08 23:03:26.020256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.020155    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw7ph" event={"ID":"3681f3f9-bf81-41c8-8caa-75b5123fe91b","Type":"ContainerDied","Data":"0acf0255e8cf31477ac0923406de6327118debb049dcb5ce7a7a3e94a60685a8"}
Jul 08 23:03:26.066439 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.066207    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tkl7k" podStartSLOduration=3.485901025 podStartE2EDuration="24.066193199s" totalImagesPullingTime="20.580292174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:26.066158763 +0000 UTC m=+24.996192743" watchObservedRunningTime="2026-07-08 23:03:26.066193199 +0000 UTC m=+24.996227173"
Jul 08 23:03:26.082612 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.082574    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g669b" podStartSLOduration=3.390271465 podStartE2EDuration="24.082560562s" totalImagesPullingTime="20.692289097s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:26.082359338 +0000 UTC m=+25.012393311" watchObservedRunningTime="2026-07-08 23:03:26.082560562 +0000 UTC m=+25.012594535"
Jul 08 23:03:26.100098 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.100062    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m6st7" podStartSLOduration=3.413515421 podStartE2EDuration="24.100047406s" totalImagesPullingTime="20.686531985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:26.09980184 +0000 UTC m=+25.029835815" watchObservedRunningTime="2026-07-08 23:03:26.100047406 +0000 UTC m=+25.030081380"
Jul 08 23:03:26.117741 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.117698    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r2qwv" podStartSLOduration=3.362759862 podStartE2EDuration="24.117682884s" totalImagesPullingTime="20.754923022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:26.117315186 +0000 UTC m=+25.047349162" watchObservedRunningTime="2026-07-08 23:03:26.117682884 +0000 UTC m=+25.047716857"
Jul 08 23:03:26.694153 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.694109    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 08 23:03:26.698763 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.698745    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:26.698880 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.698791    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:26.698880 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:26.698869    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:26.698987 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:26.698971    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:26.811686 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:26.811657    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:27.025040 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.025006    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal" event={"ID":"e71b3d9b403cf4f726d813f56bbefbdf","Type":"ContainerStarted","Data":"48caa06d2082942c4ca3f6a4fecb02052f166e37da364d0f84012c5303ef6756"}
Jul 08 23:03:27.026898 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.026870    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s" event={"ID":"903f4e45-3f42-4b97-b0e8-140303e074b5","Type":"ContainerStarted","Data":"b7f11019fafafeac1123126649557b5d2d077dfc758b63720657c137c7d6302d"}
Jul 08 23:03:27.029643 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.029625    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:03:27.029914 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.029895    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl" event={"ID":"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6","Type":"ContainerStarted","Data":"80ad60a46ae49c52ce9d9978696dd6536238c53de25baf4cc4aeb6cf214c0b56"}
Jul 08 23:03:27.039555 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.039520    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-54.ec2.internal" podStartSLOduration=24.03950705 podStartE2EDuration="24.03950705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:27.039277816 +0000 UTC m=+25.969311789" watchObservedRunningTime="2026-07-08 23:03:27.03950705 +0000 UTC m=+25.969541024"
Jul 08 23:03:27.112658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.112578    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:27.412720 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.412653    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:27.517022 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.516982    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:27.615108 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.615073    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:27.619009 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.618918    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T23:03:26.694127801Z","UUID":"eb5166e7-146e-42ff-abaa-42e7240e165f","Handler":null,"Name":"","Endpoint":""}
Jul 08 23:03:27.620566 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.620535    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 08 23:03:27.620566 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.620569    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 08 23:03:27.699402 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:27.699368    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:27.699600 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:27.699493    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:28.112908 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:28.112883    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:28.698998 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:28.698966    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:28.699181 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:28.699014    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:28.699181 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:28.699102    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:28.699256 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:28.699225    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:28.778849 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:28.778816    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:28.779316 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:28.779298    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:28.825961 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:28.825930    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:28.913434 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:28.913405    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:29.038413 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.038088    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s" event={"ID":"903f4e45-3f42-4b97-b0e8-140303e074b5","Type":"ContainerStarted","Data":"21f0ca02864d9e52ff657350f4658a2eadf418717d78dd2cd224277f005832b6"}
Jul 08 23:03:29.043170 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.042803    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:03:29.043258 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.043236    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl" event={"ID":"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6","Type":"ContainerStarted","Data":"038b6e49cf824835b14685ac47f68825997d202e38eb7c368254e901279cc8f9"}
Jul 08 23:03:29.043521 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.043498    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:29.043602 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.043581    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:29.043658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.043629    2560 scope.go:117] "RemoveContainer" containerID="31a3a2d754d80299d8192da2487ba59bf48e1452ca5380875ed8ef7c7a3e3ced"
Jul 08 23:03:29.059791 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.059751    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kf8s" podStartSLOduration=3.870278042 podStartE2EDuration="27.059735615s" totalImagesPullingTime="23.189457573s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:29.059312956 +0000 UTC m=+27.989346930" watchObservedRunningTime="2026-07-08 23:03:29.059735615 +0000 UTC m=+27.989769588"
Jul 08 23:03:29.060805 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.060790    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:29.128792 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.128733    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:29.508309 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.508275    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:29.612303 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.612266    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:29.698733 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.698702    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:29.698909 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:29.698808    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:29.716703 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.716666    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:29.905279 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:29.905203    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:30.049712 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:30.049685    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:03:30.050073 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:30.050050    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl" event={"ID":"221668dd-86f4-4e20-bc7f-12cd2d4ce3d6","Type":"ContainerStarted","Data":"4db4691dcbb4d4b16b4907e9be1cc619e2d2abf6cc5f7f78ec48210b02e40417"}
Jul 08 23:03:30.050406 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:30.050380    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:30.066162 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:30.066140    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:03:30.080045 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:30.079997    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl" podStartSLOduration=7.062343325 podStartE2EDuration="28.079984218s" totalImagesPullingTime="21.017640893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:30.079535835 +0000 UTC m=+29.009569805" watchObservedRunningTime="2026-07-08 23:03:30.079984218 +0000 UTC m=+29.010018192"
Jul 08 23:03:30.517195 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:30.517156    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:30.699039 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:30.699007    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:30.699211 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:30.699053    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:30.699211 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:30.699140    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:30.699322 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:30.699234    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:30.715370 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:30.715342    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:30.816139 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:30.816066    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:31.312786 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:31.312760    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:31.565022 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:31.564964    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:31.667522 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:31.667494    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:31.699882 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:31.699858    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:31.699989 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:31.699937    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:31.764284 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:31.764253    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:32.056204 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:32.056164    2560 generic.go:356] "Generic (PLEG): container finished" podID="3681f3f9-bf81-41c8-8caa-75b5123fe91b" containerID="0063ca2170ab5a2ab2bb4877325bcb8055055bdd36dc5c226a64206ee2705cae" exitCode=0
Jul 08 23:03:32.056360 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:32.056230    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw7ph" event={"ID":"3681f3f9-bf81-41c8-8caa-75b5123fe91b","Type":"ContainerDied","Data":"0063ca2170ab5a2ab2bb4877325bcb8055055bdd36dc5c226a64206ee2705cae"}
Jul 08 23:03:32.115997 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:32.115963    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:32.216511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:32.216480    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:32.257734 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:32.257700    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:32.257865 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:32.257815    2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 08 23:03:32.258484 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:32.258449    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r2qwv"
Jul 08 23:03:32.698507 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:32.698482    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:32.698938 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:32.698481    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:32.698938 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:32.698617    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:32.698938 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:32.698667    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:32.936316 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:32.936288    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:33.109144 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:33.109083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:33.209851 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:33.209822    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:33.505893 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:33.505866    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:33.698925 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:33.698899    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:33.699254 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:33.698994    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:33.710398 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:33.710374    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:34.065150 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:34.065061    2560 generic.go:356] "Generic (PLEG): container finished" podID="3681f3f9-bf81-41c8-8caa-75b5123fe91b" containerID="e7978e75977eedba6a24487fbb7012141fcb2d19bcf4c7b68707beade99ca895" exitCode=0
Jul 08 23:03:34.065150 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:34.065126    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw7ph" event={"ID":"3681f3f9-bf81-41c8-8caa-75b5123fe91b","Type":"ContainerDied","Data":"e7978e75977eedba6a24487fbb7012141fcb2d19bcf4c7b68707beade99ca895"}
Jul 08 23:03:34.114313 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:34.114285    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:34.315819 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:34.315584    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:34.315819 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:34.315707    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:34.316031 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:34.315838    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret podName:aa634889-de73-473e-915b-56355ad11639 nodeName:}" failed. No retries permitted until 2026-07-08 23:04:06.315824354 +0000 UTC m=+65.245858306 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret") pod "global-pull-secret-syncer-w7pd9" (UID: "aa634889-de73-473e-915b-56355ad11639") : object "kube-system"/"original-pull-secret" not registered
Jul 08 23:03:34.416802 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:34.416776    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:34.416939 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:34.416902    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:34.416979 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:34.416958    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs podName:2b56029e-de23-4191-ac78-50d4a48ca94b nodeName:}" failed. No retries permitted until 2026-07-08 23:04:06.416943557 +0000 UTC m=+65.346977508 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs") pod "network-metrics-daemon-2fss7" (UID: "2b56029e-de23-4191-ac78-50d4a48ca94b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 23:03:34.517123 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:34.517093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86pl5\" (UniqueName: \"kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5\") pod \"network-check-target-l6w79\" (UID: \"6952cf89-883f-4132-8357-52b4f055486f\") " pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:34.517261 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:34.517239    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 23:03:34.517261 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:34.517256    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 23:03:34.517327 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:34.517265    2560 projected.go:196] Error preparing data for projected volume kube-api-access-86pl5 for pod openshift-network-diagnostics/network-check-target-l6w79: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:34.517327 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:34.517314    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5 podName:6952cf89-883f-4132-8357-52b4f055486f nodeName:}" failed. No retries permitted until 2026-07-08 23:04:06.517297876 +0000 UTC m=+65.447331828 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-86pl5" (UniqueName: "kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5") pod "network-check-target-l6w79" (UID: "6952cf89-883f-4132-8357-52b4f055486f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 23:03:34.698448 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:34.698414    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:34.698621 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:34.698415    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:34.698621 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:34.698537    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:34.698621 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:34.698608    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:34.713444 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:34.713404    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:34.911652 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:34.911620    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:35.111403 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:35.111339    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:35.518749 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:35.518722    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:35.698324 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:35.698297    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:35.698493 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:35.698399    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:35.716876 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:35.716853    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:35.880259 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:35.880152    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:35.980278 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:35.980247    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:36.070674 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:36.070645    2560 generic.go:356] "Generic (PLEG): container finished" podID="3681f3f9-bf81-41c8-8caa-75b5123fe91b" containerID="fbe1bfd2ec408e702b6381835a0448ca09080c9069d63a3e8f72d052095a6a45" exitCode=0
Jul 08 23:03:36.070836 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:36.070709    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw7ph" event={"ID":"3681f3f9-bf81-41c8-8caa-75b5123fe91b","Type":"ContainerDied","Data":"fbe1bfd2ec408e702b6381835a0448ca09080c9069d63a3e8f72d052095a6a45"}
Jul 08 23:03:36.310664 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:36.310636    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:36.698455 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:36.698416    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:36.698657 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:36.698459    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:36.698657 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:36.698574    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:36.698824 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:36.698738    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:36.710566 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:36.710540    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:37.109205 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:37.109122    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:37.311739 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:37.311709    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:37.511617 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:37.511574    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:37.701832 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:37.701801    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:37.701995 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:37.701913    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:38.250277 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:38.250243    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:38.351036 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:38.351005    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:38.698852 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:38.698817    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:38.698852 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:38.698853    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:38.699091 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:38.698966    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:38.699091 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:38.699076    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:39.698342 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:39.698306    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:39.698746 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:39.698439    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:40.698478 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:40.698424    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:40.698898 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:40.698488    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:40.698898 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:40.698560    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:40.698898 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:40.698698    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:41.701207 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:41.701180    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:41.701555 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:41.701268    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:42.086982 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:42.086908    2560 generic.go:356] "Generic (PLEG): container finished" podID="3681f3f9-bf81-41c8-8caa-75b5123fe91b" containerID="3d09e30ca1fd2cc402cf4d9a89d06f5656f9038c628c72f033091642892483f8" exitCode=0
Jul 08 23:03:42.086982 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:42.086945    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw7ph" event={"ID":"3681f3f9-bf81-41c8-8caa-75b5123fe91b","Type":"ContainerDied","Data":"3d09e30ca1fd2cc402cf4d9a89d06f5656f9038c628c72f033091642892483f8"}
Jul 08 23:03:42.698375 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:42.698347    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:42.698561 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:42.698444    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:42.698561 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:42.698503    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:42.698640 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:42.698579    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:43.092141 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:43.092057    2560 generic.go:356] "Generic (PLEG): container finished" podID="3681f3f9-bf81-41c8-8caa-75b5123fe91b" containerID="bf94ea15b5a6c3aee9d4f324f6b88284e6e74cc4a09f19bd85374d752243db90" exitCode=0
Jul 08 23:03:43.092141 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:43.092126    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw7ph" event={"ID":"3681f3f9-bf81-41c8-8caa-75b5123fe91b","Type":"ContainerDied","Data":"bf94ea15b5a6c3aee9d4f324f6b88284e6e74cc4a09f19bd85374d752243db90"}
Jul 08 23:03:43.702034 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:43.702007    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:43.702174 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:43.702099    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:44.097115 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:44.097041    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw7ph" event={"ID":"3681f3f9-bf81-41c8-8caa-75b5123fe91b","Type":"ContainerStarted","Data":"4ad34b5a23524afe03239bd1adf8716a08429ca0cf70969668ab78be07a2a9cc"}
Jul 08 23:03:44.120120 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:44.120080    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vw7ph" podStartSLOduration=8.249086985 podStartE2EDuration="42.120066948s" totalImagesPullingTime="33.870979963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:03:44.118782266 +0000 UTC m=+43.048816242" watchObservedRunningTime="2026-07-08 23:03:44.120066948 +0000 UTC m=+43.050100923"
Jul 08 23:03:44.698599 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:44.698562    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:44.698599 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:44.698584    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:44.698797 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:44.698678    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:44.698834 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:44.698799    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:45.699213 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:45.699177    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:45.699688 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:45.699281    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:46.699014 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:46.698981    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:46.699248 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:46.699086    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:46.699248 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:46.699140    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:46.699248 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:46.699224    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:47.698542 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:47.698508    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:47.698740 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:47.698605    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:48.204354 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:48.204316    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:48.296801 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:48.296771    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:48.388488 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:48.388440    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:48.489999 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:48.489922    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:48.588970 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:48.588940    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:48.698581 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:48.698550    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:48.698581 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:48.698574    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:48.698774 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:48.698650    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:48.698820 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:48.698799    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:48.860669 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:48.860590    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:49.059665 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.059630    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:49.217968 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.217931    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l6w79"]
Jul 08 23:03:49.218383 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.218026    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:49.218383 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:49.218114    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:49.222046 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.222022    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2fss7"]
Jul 08 23:03:49.222160 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.222105    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:49.222231 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:49.222184    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:49.236359 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.236328    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w7pd9"]
Jul 08 23:03:49.236511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.236417    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:49.236558 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:49.236508    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:49.272989 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.272960    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:49.371963 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.371887    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:49.432927 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.432898    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:49.615812 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.615779    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:49.716194 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:49.716166    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:50.698914 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:50.698886    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:50.699334 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:50.698886    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:50.699334 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:50.698992    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:50.699334 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:50.698886    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:50.699334 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:50.699068    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:50.699334 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:50.699166    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:52.698597 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:52.698565    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:52.698597 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:52.698595    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:52.699007 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:52.698565    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:52.699007 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:52.698670    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6w79" podUID="6952cf89-883f-4132-8357-52b4f055486f"
Jul 08 23:03:52.699007 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:52.698742    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fss7" podUID="2b56029e-de23-4191-ac78-50d4a48ca94b"
Jul 08 23:03:52.699007 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:52.698827    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7pd9" podUID="aa634889-de73-473e-915b-56355ad11639"
Jul 08 23:03:53.881420 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.881344    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-54.ec2.internal" event="NodeReady"
Jul 08 23:03:53.881839 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.881453    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 08 23:03:53.920684 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.920654    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l799w"]
Jul 08 23:03:53.922598 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.922578    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:53.923839 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.923817    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-56gmw"]
Jul 08 23:03:53.924795 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.924775    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 08 23:03:53.924861 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.924775    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 08 23:03:53.924895 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.924883    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2zvtt\""
Jul 08 23:03:53.925640 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.925627    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:53.927671 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.927655    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 08 23:03:53.927992 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.927977    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 08 23:03:53.928062 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.928049    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 08 23:03:53.928445 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.928431    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 08 23:03:53.928550 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.928536    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ntpx9\""
Jul 08 23:03:53.933686 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.933668    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l799w"]
Jul 08 23:03:53.938002 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.937984    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-56gmw"]
Jul 08 23:03:53.942715 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.942697    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-smf48"]
Jul 08 23:03:53.944429 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.944414    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-smf48"
Jul 08 23:03:53.948108 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.948081    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xlh5l\""
Jul 08 23:03:53.948209 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.948139    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 08 23:03:53.948209 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.948139    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 08 23:03:53.948209 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.948169    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 08 23:03:53.956634 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.956614    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-smf48"]
Jul 08 23:03:53.960079 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.960060    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/749c95ca-a7ba-41bf-a894-3fcf44d9de75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:53.960175 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.960097    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/749c95ca-a7ba-41bf-a894-3fcf44d9de75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:53.960175 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.960114    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-config-volume\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:53.960175 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.960136    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/749c95ca-a7ba-41bf-a894-3fcf44d9de75-data-volume\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:53.960305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.960179    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-metrics-tls\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:53.960305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.960213    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/749c95ca-a7ba-41bf-a894-3fcf44d9de75-crio-socket\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:53.960305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.960230    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql9vg\" (UniqueName: \"kubernetes.io/projected/749c95ca-a7ba-41bf-a894-3fcf44d9de75-kube-api-access-ql9vg\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:53.960305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.960247    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgz9g\" (UniqueName: \"kubernetes.io/projected/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-kube-api-access-jgz9g\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:53.960305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:53.960272    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-tmp-dir\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:54.041327 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.041295    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:54.061509 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061460    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-tmp-dir\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:54.061626 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061519    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9djp\" (UniqueName: \"kubernetes.io/projected/efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb-kube-api-access-c9djp\") pod \"ingress-canary-smf48\" (UID: \"efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb\") " pod="openshift-ingress-canary/ingress-canary-smf48"
Jul 08 23:03:54.061626 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061537    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb-cert\") pod \"ingress-canary-smf48\" (UID: \"efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb\") " pod="openshift-ingress-canary/ingress-canary-smf48"
Jul 08 23:03:54.061626 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061571    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/749c95ca-a7ba-41bf-a894-3fcf44d9de75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:54.061785 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061637    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/749c95ca-a7ba-41bf-a894-3fcf44d9de75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:54.061785 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061658    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-config-volume\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:54.061785 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061689    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/749c95ca-a7ba-41bf-a894-3fcf44d9de75-data-volume\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:54.061785 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061706    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-metrics-tls\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:54.061785 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061724    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/749c95ca-a7ba-41bf-a894-3fcf44d9de75-crio-socket\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:54.061785 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061748    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql9vg\" (UniqueName: \"kubernetes.io/projected/749c95ca-a7ba-41bf-a894-3fcf44d9de75-kube-api-access-ql9vg\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:54.061785 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061774    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgz9g\" (UniqueName: \"kubernetes.io/projected/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-kube-api-access-jgz9g\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:54.062113 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061813    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-tmp-dir\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:54.062113 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.061955    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/749c95ca-a7ba-41bf-a894-3fcf44d9de75-crio-socket\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:54.062113 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.062078    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/749c95ca-a7ba-41bf-a894-3fcf44d9de75-data-volume\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:54.062113 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.062084    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/749c95ca-a7ba-41bf-a894-3fcf44d9de75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:54.062315 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.062298    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-config-volume\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:54.065699 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.065677    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-metrics-tls\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:54.065791 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.065680    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/749c95ca-a7ba-41bf-a894-3fcf44d9de75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:54.069770 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.069749    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgz9g\" (UniqueName: \"kubernetes.io/projected/e1cec76f-e1c9-4ad1-90d7-e0645d42e834-kube-api-access-jgz9g\") pod \"dns-default-l799w\" (UID: \"e1cec76f-e1c9-4ad1-90d7-e0645d42e834\") " pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:54.069890 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.069789    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql9vg\" (UniqueName: \"kubernetes.io/projected/749c95ca-a7ba-41bf-a894-3fcf44d9de75-kube-api-access-ql9vg\") pod \"insights-runtime-extractor-56gmw\" (UID: \"749c95ca-a7ba-41bf-a894-3fcf44d9de75\") " pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:54.162612 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.162525    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9djp\" (UniqueName: \"kubernetes.io/projected/efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb-kube-api-access-c9djp\") pod \"ingress-canary-smf48\" (UID: \"efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb\") " pod="openshift-ingress-canary/ingress-canary-smf48"
Jul 08 23:03:54.162612 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.162559    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb-cert\") pod \"ingress-canary-smf48\" (UID: \"efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb\") " pod="openshift-ingress-canary/ingress-canary-smf48"
Jul 08 23:03:54.164887 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.164864    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb-cert\") pod \"ingress-canary-smf48\" (UID: \"efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb\") " pod="openshift-ingress-canary/ingress-canary-smf48"
Jul 08 23:03:54.170361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.170342    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9djp\" (UniqueName: \"kubernetes.io/projected/efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb-kube-api-access-c9djp\") pod \"ingress-canary-smf48\" (UID: \"efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb\") " pod="openshift-ingress-canary/ingress-canary-smf48"
Jul 08 23:03:54.233255 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.233216    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l799w"
Jul 08 23:03:54.239843 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.239820    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-56gmw"
Jul 08 23:03:54.254747 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.254726    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-smf48"
Jul 08 23:03:54.385676 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.385648    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l799w"]
Jul 08 23:03:54.388677 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.388649    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-smf48"]
Jul 08 23:03:54.391247 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.391053    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-56gmw"]
Jul 08 23:03:54.393264 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:54.393239    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb5cd57_0c10_4b3e_9d88_0bdf9d2f22bb.slice/crio-1e2903908875caee278343619cd63fa3089fddcee7c9bd41d8cf213e87d96752 WatchSource:0}: Error finding container 1e2903908875caee278343619cd63fa3089fddcee7c9bd41d8cf213e87d96752: Status 404 returned error can't find the container with id 1e2903908875caee278343619cd63fa3089fddcee7c9bd41d8cf213e87d96752
Jul 08 23:03:54.394143 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:54.394118    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749c95ca_a7ba_41bf_a894_3fcf44d9de75.slice/crio-ecfbf476c200d1fe8b5519b59a570b7dd31b4a094e3a8476ec778359546d7bcb WatchSource:0}: Error finding container ecfbf476c200d1fe8b5519b59a570b7dd31b4a094e3a8476ec778359546d7bcb: Status 404 returned error can't find the container with id ecfbf476c200d1fe8b5519b59a570b7dd31b4a094e3a8476ec778359546d7bcb
Jul 08 23:03:54.698654 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.698624    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:03:54.698808 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.698671    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:03:54.698808 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.698634    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:03:54.701416 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.701387    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 23:03:54.702374 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.702350    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 23:03:54.702521 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.702353    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8bt8r\""
Jul 08 23:03:54.702521 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.702392    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 23:03:54.702685 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.702520    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 23:03:54.702685 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:54.702644    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f6f6s\""
Jul 08 23:03:55.123125 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:55.123042    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56gmw" event={"ID":"749c95ca-a7ba-41bf-a894-3fcf44d9de75","Type":"ContainerStarted","Data":"ecfbf476c200d1fe8b5519b59a570b7dd31b4a094e3a8476ec778359546d7bcb"}
Jul 08 23:03:55.124938 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:55.124911    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l799w" event={"ID":"e1cec76f-e1c9-4ad1-90d7-e0645d42e834","Type":"ContainerStarted","Data":"362f81df95b26791f5c2e0ec1531617d18d770cfa446ea02630e92c712e51984"}
Jul 08 23:03:55.127349 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:55.127324    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-smf48" event={"ID":"efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb","Type":"ContainerStarted","Data":"1e2903908875caee278343619cd63fa3089fddcee7c9bd41d8cf213e87d96752"}
Jul 08 23:03:57.062672 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.062615    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:57.100345 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.097869    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"]
Jul 08 23:03:57.106065 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.106036    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.109039 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.109009    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"]
Jul 08 23:03:57.109181 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.109044    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 08 23:03:57.109181 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.109070    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 08 23:03:57.109300 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.109186    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 08 23:03:57.109300 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.109196    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bcfpj\""
Jul 08 23:03:57.109411 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.109317    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 08 23:03:57.110115 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.110091    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 08 23:03:57.124639 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.124599    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:57.187581 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.187545    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7089b439-fd61-4b8a-8cd1-bbf86c74809d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.187841 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.187586    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7089b439-fd61-4b8a-8cd1-bbf86c74809d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.187841 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.187675    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7089b439-fd61-4b8a-8cd1-bbf86c74809d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.187841 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.187698    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxlbn\" (UniqueName: \"kubernetes.io/projected/7089b439-fd61-4b8a-8cd1-bbf86c74809d-kube-api-access-mxlbn\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.289022 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.288982    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7089b439-fd61-4b8a-8cd1-bbf86c74809d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.289022 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.289017    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7089b439-fd61-4b8a-8cd1-bbf86c74809d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.289260 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.289071    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7089b439-fd61-4b8a-8cd1-bbf86c74809d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.289260 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.289093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxlbn\" (UniqueName: \"kubernetes.io/projected/7089b439-fd61-4b8a-8cd1-bbf86c74809d-kube-api-access-mxlbn\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.289260 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:57.289163    2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 08 23:03:57.289260 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:03:57.289242    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7089b439-fd61-4b8a-8cd1-bbf86c74809d-prometheus-operator-tls podName:7089b439-fd61-4b8a-8cd1-bbf86c74809d nodeName:}" failed. No retries permitted until 2026-07-08 23:03:57.789220051 +0000 UTC m=+56.719254027 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7089b439-fd61-4b8a-8cd1-bbf86c74809d-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-lkpl9" (UID: "7089b439-fd61-4b8a-8cd1-bbf86c74809d") : secret "prometheus-operator-tls" not found
Jul 08 23:03:57.290182 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.290153    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7089b439-fd61-4b8a-8cd1-bbf86c74809d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.292642 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.292618    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7089b439-fd61-4b8a-8cd1-bbf86c74809d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.300135 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.300110    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxlbn\" (UniqueName: \"kubernetes.io/projected/7089b439-fd61-4b8a-8cd1-bbf86c74809d-kube-api-access-mxlbn\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.451612 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.451575    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:57.500266 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.500175    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:57.656925 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.656892    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:57.793790 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.793699    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7089b439-fd61-4b8a-8cd1-bbf86c74809d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:57.796206 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:57.796179    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7089b439-fd61-4b8a-8cd1-bbf86c74809d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lkpl9\" (UID: \"7089b439-fd61-4b8a-8cd1-bbf86c74809d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:58.018097 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:58.018056    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"
Jul 08 23:03:59.248372 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:59.248333    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:59.352847 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:59.352748    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:03:59.428999 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:03:59.428921    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9"]
Jul 08 23:03:59.433427 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:03:59.433400    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7089b439_fd61_4b8a_8cd1_bbf86c74809d.slice/crio-82df770901492a5a3e221f39718533e67a8a7be88d097d85aefefa791fe59c79 WatchSource:0}: Error finding container 82df770901492a5a3e221f39718533e67a8a7be88d097d85aefefa791fe59c79: Status 404 returned error can't find the container with id 82df770901492a5a3e221f39718533e67a8a7be88d097d85aefefa791fe59c79
Jul 08 23:04:00.145972 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:00.145943    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-smf48" event={"ID":"efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb","Type":"ContainerStarted","Data":"50278b9b17c3f9f1a8991c8d7d2e0275ead190609d1fc842c694bea92f85cacc"}
Jul 08 23:04:00.149826 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:00.149794    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9" event={"ID":"7089b439-fd61-4b8a-8cd1-bbf86c74809d","Type":"ContainerStarted","Data":"82df770901492a5a3e221f39718533e67a8a7be88d097d85aefefa791fe59c79"}
Jul 08 23:04:00.152275 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:00.152220    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56gmw" event={"ID":"749c95ca-a7ba-41bf-a894-3fcf44d9de75","Type":"ContainerStarted","Data":"1425b86ce3f01031f25fb5dac398efffe5cd22a46aa2b666e8bcc4c157738392"}
Jul 08 23:04:00.154291 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:00.154267    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l799w" event={"ID":"e1cec76f-e1c9-4ad1-90d7-e0645d42e834","Type":"ContainerStarted","Data":"25b4b16789d7018eaef19d2f40474ec09557451b98e934ee499be1de10e45165"}
Jul 08 23:04:00.154396 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:00.154314    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l799w" event={"ID":"e1cec76f-e1c9-4ad1-90d7-e0645d42e834","Type":"ContainerStarted","Data":"bdb327f0b1f0e229505dfa101c27642909088a1fd03f4e642ac964935b382eca"}
Jul 08 23:04:00.154461 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:00.154441    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l799w"
Jul 08 23:04:00.160575 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:00.160487    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-smf48" podStartSLOduration=2.2420272629999998 podStartE2EDuration="7.160454546s" totalImagesPullingTime="4.918427283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:00.159902029 +0000 UTC m=+59.089936003" watchObservedRunningTime="2026-07-08 23:04:00.160454546 +0000 UTC m=+59.090488514"
Jul 08 23:04:00.179440 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:00.179363    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l799w" podStartSLOduration=2.2769575140000002 podStartE2EDuration="7.179347051s" totalImagesPullingTime="4.902389537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:00.179077806 +0000 UTC m=+59.109111773" watchObservedRunningTime="2026-07-08 23:04:00.179347051 +0000 UTC m=+59.109381026"
Jul 08 23:04:01.074890 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:01.074841    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:01.126967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:01.126053    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:01.162965 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:01.162824    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9" event={"ID":"7089b439-fd61-4b8a-8cd1-bbf86c74809d","Type":"ContainerStarted","Data":"b1ee2ad70610f560dd8b4b4562af37e5f53be97cfc605fb970c1fafa268796f5"}
Jul 08 23:04:01.162965 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:01.162919    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9" event={"ID":"7089b439-fd61-4b8a-8cd1-bbf86c74809d","Type":"ContainerStarted","Data":"a9c59b058380a62fbd2499fa1972ab4e72ef277909e7ebdd7feb2cc795e153dd"}
Jul 08 23:04:01.165948 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:01.165856    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56gmw" event={"ID":"749c95ca-a7ba-41bf-a894-3fcf44d9de75","Type":"ContainerStarted","Data":"5d6e466142ae2bdfdf87223f0c92a26348372b3bb249190348dbc7a2822fe25f"}
Jul 08 23:04:01.180153 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:01.180042    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lkpl9" podStartSLOduration=2.6918481720000003 podStartE2EDuration="4.180002999s" totalImagesPullingTime="1.488154827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:01.179265703 +0000 UTC m=+60.109299677" watchObservedRunningTime="2026-07-08 23:04:01.180002999 +0000 UTC m=+60.110036975"
Jul 08 23:04:01.202027 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:01.201995    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:02.067972 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:02.067943    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fcsjl"
Jul 08 23:04:02.187478 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:02.187437    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:02.245485 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:02.245436    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:03.173514 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.173485    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56gmw" event={"ID":"749c95ca-a7ba-41bf-a894-3fcf44d9de75","Type":"ContainerStarted","Data":"b43bc74a79af33caf2a3968273aefd584ac351c31fdc3c5e7dd8532f11a61deb"}
Jul 08 23:04:03.202883 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.202858    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:03.203280 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.203033    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-56gmw" podStartSLOduration=1.797757251 podStartE2EDuration="10.20301974s" totalImagesPullingTime="8.405262489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:03.202703102 +0000 UTC m=+62.132737077" watchObservedRunningTime="2026-07-08 23:04:03.20301974 +0000 UTC m=+62.133053749"
Jul 08 23:04:03.408059 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.407990    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:03.516709 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.516684    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wdw8h"]
Jul 08 23:04:03.518445 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.518431    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.520975 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.520955    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sc5qc\""
Jul 08 23:04:03.521089 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.520986    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 08 23:04:03.521423 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.521410    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 08 23:04:03.521614 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.521602    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 08 23:04:03.533636 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.533612    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:03.617915 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.617869    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:03.639111 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.639087    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-wtmp\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.639213 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.639115    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkvrb\" (UniqueName: \"kubernetes.io/projected/f05ce760-1a5c-45ca-a213-752db93a15ee-kube-api-access-hkvrb\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.639213 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.639147    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f05ce760-1a5c-45ca-a213-752db93a15ee-metrics-client-ca\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.639213 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.639171    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.639317 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.639225    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f05ce760-1a5c-45ca-a213-752db93a15ee-sys\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.639317 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.639239    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-tls\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.639317 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.639254    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.639317 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.639272    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f05ce760-1a5c-45ca-a213-752db93a15ee-root\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.639317 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.639287    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-textfile\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.718843 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.718821    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:03.739769 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.739744    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-wtmp\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.739847 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.739770    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkvrb\" (UniqueName: \"kubernetes.io/projected/f05ce760-1a5c-45ca-a213-752db93a15ee-kube-api-access-hkvrb\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.739847 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.739794    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f05ce760-1a5c-45ca-a213-752db93a15ee-metrics-client-ca\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.739932 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.739900    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.739970 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.739943    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f05ce760-1a5c-45ca-a213-752db93a15ee-sys\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.739970 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.739961    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-tls\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.740061 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.739976    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.740061 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.739997    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f05ce760-1a5c-45ca-a213-752db93a15ee-root\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.740061 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.740018    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-textfile\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.740061 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.739905    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-wtmp\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.740061 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.740045    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f05ce760-1a5c-45ca-a213-752db93a15ee-sys\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.740276 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.740102    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f05ce760-1a5c-45ca-a213-752db93a15ee-root\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.740276 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:04:03.740158    2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 08 23:04:03.740276 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:04:03.740208    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-tls podName:f05ce760-1a5c-45ca-a213-752db93a15ee nodeName:}" failed. No retries permitted until 2026-07-08 23:04:04.240191643 +0000 UTC m=+63.170225595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-tls") pod "node-exporter-wdw8h" (UID: "f05ce760-1a5c-45ca-a213-752db93a15ee") : secret "node-exporter-tls" not found
Jul 08 23:04:03.740438 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.740306    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-textfile\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.740438 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.740399    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.740558 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.740483    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f05ce760-1a5c-45ca-a213-752db93a15ee-metrics-client-ca\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.742334 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.742315    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.747909 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.747892    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkvrb\" (UniqueName: \"kubernetes.io/projected/f05ce760-1a5c-45ca-a213-752db93a15ee-kube-api-access-hkvrb\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:03.826861 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.826819    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:03.991357 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:03.991274    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:04.088625 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.088595    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:04.188688 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.188660    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:04.243888 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.243832    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-tls\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:04.245937 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.245919    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f05ce760-1a5c-45ca-a213-752db93a15ee-node-exporter-tls\") pod \"node-exporter-wdw8h\" (UID: \"f05ce760-1a5c-45ca-a213-752db93a15ee\") " pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:04.427327 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.427293    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wdw8h"
Jul 08 23:04:04.454100 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.454074    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:04.465286 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:04:04.465255    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05ce760_1a5c_45ca_a213_752db93a15ee.slice/crio-83595cef36d483a6398dd2341440a777c4036b2cf4f0bdbdee503ae3e2ba56f2 WatchSource:0}: Error finding container 83595cef36d483a6398dd2341440a777c4036b2cf4f0bdbdee503ae3e2ba56f2: Status 404 returned error can't find the container with id 83595cef36d483a6398dd2341440a777c4036b2cf4f0bdbdee503ae3e2ba56f2
Jul 08 23:04:04.551232 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.551162    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:04.552586 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.552563    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 23:04:04.582210 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.582190    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 23:04:04.582329 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.582314    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.585015 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.584996    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 08 23:04:04.585124 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.585026    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 08 23:04:04.585319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.585304    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 08 23:04:04.585475 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.585437    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 08 23:04:04.585581 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.585485    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 08 23:04:04.585581 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.585534    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-b4578\""
Jul 08 23:04:04.585581 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.585537    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 08 23:04:04.585720 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.585582    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 08 23:04:04.585720 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.585605    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 08 23:04:04.585720 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.585680    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 08 23:04:04.647233 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-web-config\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647377 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647237    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h64p\" (UniqueName: \"kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-kube-api-access-7h64p\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647377 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647283    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647377 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647318    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647377 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647349    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647377 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647366    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647565 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647396    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647565 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647426    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647565 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647445    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647565 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647479    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647565 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647514    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-out\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647565 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647531    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.647565 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.647545    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.748459 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748430    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.748607 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748484    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-out\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.748607 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748510    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.748607 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748532    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.748607 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748589    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-web-config\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.748818 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748616    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h64p\" (UniqueName: \"kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-kube-api-access-7h64p\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.748818 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748670    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.748818 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748702    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.748818 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748741    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.748818 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748768    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.748818 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748802    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.749089 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748826    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.749089 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.748855    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.749190 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.749155    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.749484 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.749440    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.750305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.750265    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.751328 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.751305    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-out\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.751616 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.751595    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.751747 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.751726    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-web-config\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.751893 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.751870    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.752000 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.751921    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.754529 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.752337    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.754529 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.753205    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.754529 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.753759    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.755366 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.755347    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.756347 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.756329    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h64p\" (UniqueName: \"kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-kube-api-access-7h64p\") pod \"alertmanager-main-0\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:04.765260 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.765241    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:04.891634 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:04.891566    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:04:05.047866 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.047837    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 23:04:05.051184 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:04:05.051156    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e229b86_1eee_4335_9cd1_a3a641ea9d50.slice/crio-ad286a7401fa867a09e164fa3c0f2cb4397986d192ba46a4527359f65aa5ddb0 WatchSource:0}: Error finding container ad286a7401fa867a09e164fa3c0f2cb4397986d192ba46a4527359f65aa5ddb0: Status 404 returned error can't find the container with id ad286a7401fa867a09e164fa3c0f2cb4397986d192ba46a4527359f65aa5ddb0
Jul 08 23:04:05.181518 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.181486    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wdw8h" event={"ID":"f05ce760-1a5c-45ca-a213-752db93a15ee","Type":"ContainerStarted","Data":"83595cef36d483a6398dd2341440a777c4036b2cf4f0bdbdee503ae3e2ba56f2"}
Jul 08 23:04:05.182937 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.182906    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerStarted","Data":"ad286a7401fa867a09e164fa3c0f2cb4397986d192ba46a4527359f65aa5ddb0"}
Jul 08 23:04:05.445395 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.445328    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-c4cd44f8-6krxk"]
Jul 08 23:04:05.478690 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.478653    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-c4cd44f8-6krxk"]
Jul 08 23:04:05.478850 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.478791    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.481086 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.481062    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 08 23:04:05.481226 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.481109    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 08 23:04:05.481335 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.481315    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q62w5\""
Jul 08 23:04:05.481335 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.481327    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f86qml2ns10cr\""
Jul 08 23:04:05.481519 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.481361    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 08 23:04:05.481519 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.481510    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 08 23:04:05.481647 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.481631    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 08 23:04:05.555755 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.555727    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.555909 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.555771    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.555909 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.555888    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-grpc-tls\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.556012 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.555960    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxf6d\" (UniqueName: \"kubernetes.io/projected/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-kube-api-access-kxf6d\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.556012 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.555993    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-metrics-client-ca\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.556103 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.556026    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-tls\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.556103 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.556048    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.556196 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.556140    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.657050 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.657019    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-tls\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.657161 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.657142    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.657222 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.657199    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.657282 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.657251    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.657335 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.657282    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.657390 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.657346    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-grpc-tls\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.657443 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.657392    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxf6d\" (UniqueName: \"kubernetes.io/projected/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-kube-api-access-kxf6d\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.657443 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.657421    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-metrics-client-ca\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.658169 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.658108    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-metrics-client-ca\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.660044 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.659965    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-tls\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.660148 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.660047    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.660148 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.660108    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.660270 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.660189    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.660748 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.660729    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-grpc-tls\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.660829 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.660785    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.665403 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.665386    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxf6d\" (UniqueName: \"kubernetes.io/projected/b5a735f9-fd2e-4a5b-8407-0a4f8af80090-kube-api-access-kxf6d\") pod \"thanos-querier-c4cd44f8-6krxk\" (UID: \"b5a735f9-fd2e-4a5b-8407-0a4f8af80090\") " pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.790770 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.790747    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:05.938736 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:05.938671    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-c4cd44f8-6krxk"]
Jul 08 23:04:05.942791 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:04:05.942765    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a735f9_fd2e_4a5b_8407_0a4f8af80090.slice/crio-92b6b81ed25b10f560681a7c14c96c3390644b9f2578a6ce6f89dbdaa162c421 WatchSource:0}: Error finding container 92b6b81ed25b10f560681a7c14c96c3390644b9f2578a6ce6f89dbdaa162c421: Status 404 returned error can't find the container with id 92b6b81ed25b10f560681a7c14c96c3390644b9f2578a6ce6f89dbdaa162c421
Jul 08 23:04:06.191564 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.191524    2560 generic.go:356] "Generic (PLEG): container finished" podID="f05ce760-1a5c-45ca-a213-752db93a15ee" containerID="b797a8abfb69b40b9c3198e9945a1e543891aa5c355d88448539f1473f672770" exitCode=0
Jul 08 23:04:06.191765 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.191622    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wdw8h" event={"ID":"f05ce760-1a5c-45ca-a213-752db93a15ee","Type":"ContainerDied","Data":"b797a8abfb69b40b9c3198e9945a1e543891aa5c355d88448539f1473f672770"}
Jul 08 23:04:06.192964 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.192943    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk" event={"ID":"b5a735f9-fd2e-4a5b-8407-0a4f8af80090","Type":"ContainerStarted","Data":"92b6b81ed25b10f560681a7c14c96c3390644b9f2578a6ce6f89dbdaa162c421"}
Jul 08 23:04:06.324663 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.324619    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:06.362967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.362943    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:04:06.365243 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.365226    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 23:04:06.375695 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.375665    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa634889-de73-473e-915b-56355ad11639-original-pull-secret\") pod \"global-pull-secret-syncer-w7pd9\" (UID: \"aa634889-de73-473e-915b-56355ad11639\") " pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:04:06.415345 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.415325    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7pd9"
Jul 08 23:04:06.425411 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.425352    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:06.463931 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.463806    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:04:06.466723 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.466596    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 23:04:06.477275 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.477253    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b56029e-de23-4191-ac78-50d4a48ca94b-metrics-certs\") pod \"network-metrics-daemon-2fss7\" (UID: \"2b56029e-de23-4191-ac78-50d4a48ca94b\") " pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:04:06.525972 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.525943    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:06.541215 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.541186    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w7pd9"]
Jul 08 23:04:06.544549 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:04:06.544526    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa634889_de73_473e_915b_56355ad11639.slice/crio-e970a5556b55c064dff170d9925aeab456cd940e98779ad79c135cb6eed176d9 WatchSource:0}: Error finding container e970a5556b55c064dff170d9925aeab456cd940e98779ad79c135cb6eed176d9: Status 404 returned error can't find the container with id e970a5556b55c064dff170d9925aeab456cd940e98779ad79c135cb6eed176d9
Jul 08 23:04:06.564972 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.564942    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86pl5\" (UniqueName: \"kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5\") pod \"network-check-target-l6w79\" (UID: \"6952cf89-883f-4132-8357-52b4f055486f\") " pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:04:06.567402 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.567384    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 23:04:06.577455 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.577430    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 23:04:06.587613 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.587585    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86pl5\" (UniqueName: \"kubernetes.io/projected/6952cf89-883f-4132-8357-52b4f055486f-kube-api-access-86pl5\") pod \"network-check-target-l6w79\" (UID: \"6952cf89-883f-4132-8357-52b4f055486f\") " pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:04:06.625838 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.625796    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:06.711448 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.711362    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8bt8r\""
Jul 08 23:04:06.719483 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.719448    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fss7"
Jul 08 23:04:06.721814 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.721794    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f6f6s\""
Jul 08 23:04:06.726077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.726055    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:06.729791 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.729729    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:04:06.851554 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.851462    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2fss7"]
Jul 08 23:04:06.855010 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:04:06.854978    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b56029e_de23_4191_ac78_50d4a48ca94b.slice/crio-536af3fc39a1c42132b3551aef34e18ee8d21fabe832e8d78af3a8671783c450 WatchSource:0}: Error finding container 536af3fc39a1c42132b3551aef34e18ee8d21fabe832e8d78af3a8671783c450: Status 404 returned error can't find the container with id 536af3fc39a1c42132b3551aef34e18ee8d21fabe832e8d78af3a8671783c450
Jul 08 23:04:06.871772 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:06.871748    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l6w79"]
Jul 08 23:04:06.874764 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:04:06.874729    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6952cf89_883f_4132_8357_52b4f055486f.slice/crio-351977519c5fd8b41f6594a85ab3ec9a14bb316d65bbc892633be51067c60663 WatchSource:0}: Error finding container 351977519c5fd8b41f6594a85ab3ec9a14bb316d65bbc892633be51067c60663: Status 404 returned error can't find the container with id 351977519c5fd8b41f6594a85ab3ec9a14bb316d65bbc892633be51067c60663
Jul 08 23:04:07.199866 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.199833    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wdw8h" event={"ID":"f05ce760-1a5c-45ca-a213-752db93a15ee","Type":"ContainerStarted","Data":"098b57ca4315d5c336d028dae02c5f95565b7ca0f61b4ee7c6623433caa83132"}
Jul 08 23:04:07.200036 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.199902    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wdw8h" event={"ID":"f05ce760-1a5c-45ca-a213-752db93a15ee","Type":"ContainerStarted","Data":"5500cbf38752d6d3f6806eaf1648a084982656fc72779a04997b72608a5d0a70"}
Jul 08 23:04:07.201209 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.201172    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2fss7" event={"ID":"2b56029e-de23-4191-ac78-50d4a48ca94b","Type":"ContainerStarted","Data":"536af3fc39a1c42132b3551aef34e18ee8d21fabe832e8d78af3a8671783c450"}
Jul 08 23:04:07.203817 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.203788    2560 generic.go:356] "Generic (PLEG): container finished" podID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerID="36c56140a9836c5be7a58f673bb2075e6efb52c0fc715600b5c56d6754c0ab3e" exitCode=0
Jul 08 23:04:07.204148 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.204034    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerDied","Data":"36c56140a9836c5be7a58f673bb2075e6efb52c0fc715600b5c56d6754c0ab3e"}
Jul 08 23:04:07.207146 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.207114    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l6w79" event={"ID":"6952cf89-883f-4132-8357-52b4f055486f","Type":"ContainerStarted","Data":"351977519c5fd8b41f6594a85ab3ec9a14bb316d65bbc892633be51067c60663"}
Jul 08 23:04:07.210605 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.210581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w7pd9" event={"ID":"aa634889-de73-473e-915b-56355ad11639","Type":"ContainerStarted","Data":"e970a5556b55c064dff170d9925aeab456cd940e98779ad79c135cb6eed176d9"}
Jul 08 23:04:07.242055 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.241964    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wdw8h" podStartSLOduration=3.006323039 podStartE2EDuration="4.241948349s" totalImagesPullingTime="1.171727483s" totalInitContainerRuntime="63.897827ms" isStatefulPod=true podCreationTimestamp="2026-07-08 23:04:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:07.217992749 +0000 UTC m=+66.148026724" watchObservedRunningTime="2026-07-08 23:04:07.241948349 +0000 UTC m=+66.171982328"
Jul 08 23:04:07.863625 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.863292    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-86b868f65f-pbjg2"]
Jul 08 23:04:07.866292 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.866251    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:07.869220 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.869159    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ld7st\""
Jul 08 23:04:07.869782 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.869727    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 08 23:04:07.870256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.870238    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 08 23:04:07.870462 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.870437    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3h5si66cc7cj1\""
Jul 08 23:04:07.870615 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.870574    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 08 23:04:07.870690 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.870675    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 08 23:04:07.876495 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.876423    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86b868f65f-pbjg2"]
Jul 08 23:04:07.978771 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.978735    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ee533351-5f8b-461e-b284-af522067952f-secret-metrics-server-tls\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:07.978923 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.978774    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ee533351-5f8b-461e-b284-af522067952f-metrics-server-audit-profiles\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:07.978923 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.978812    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ee533351-5f8b-461e-b284-af522067952f-audit-log\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:07.978923 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.978867    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grvff\" (UniqueName: \"kubernetes.io/projected/ee533351-5f8b-461e-b284-af522067952f-kube-api-access-grvff\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:07.978923 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.978911    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ee533351-5f8b-461e-b284-af522067952f-secret-metrics-server-client-certs\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:07.979138 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.978942    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee533351-5f8b-461e-b284-af522067952f-client-ca-bundle\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:07.979138 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:07.979045    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee533351-5f8b-461e-b284-af522067952f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.079566 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.079511    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee533351-5f8b-461e-b284-af522067952f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.079749 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.079601    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ee533351-5f8b-461e-b284-af522067952f-secret-metrics-server-tls\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.079749 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.079632    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ee533351-5f8b-461e-b284-af522067952f-metrics-server-audit-profiles\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.079749 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.079692    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ee533351-5f8b-461e-b284-af522067952f-audit-log\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.079749 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.079727    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grvff\" (UniqueName: \"kubernetes.io/projected/ee533351-5f8b-461e-b284-af522067952f-kube-api-access-grvff\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.079956 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.079759    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ee533351-5f8b-461e-b284-af522067952f-secret-metrics-server-client-certs\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.079956 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.079784    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee533351-5f8b-461e-b284-af522067952f-client-ca-bundle\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.080630 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.080604    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ee533351-5f8b-461e-b284-af522067952f-audit-log\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.081357 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.081332    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee533351-5f8b-461e-b284-af522067952f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.081690 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.081667    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ee533351-5f8b-461e-b284-af522067952f-metrics-server-audit-profiles\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.084375 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.084331    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ee533351-5f8b-461e-b284-af522067952f-secret-metrics-server-client-certs\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.085814 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.085782    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee533351-5f8b-461e-b284-af522067952f-client-ca-bundle\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.085905 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.085786    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ee533351-5f8b-461e-b284-af522067952f-secret-metrics-server-tls\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.089361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.089341    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grvff\" (UniqueName: \"kubernetes.io/projected/ee533351-5f8b-461e-b284-af522067952f-kube-api-access-grvff\") pod \"metrics-server-86b868f65f-pbjg2\" (UID: \"ee533351-5f8b-461e-b284-af522067952f\") " pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.180763 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.180730    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:08.198027 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.197996    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:08.431681 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.431575    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:08.969033 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:08.968925    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-86b868f65f-pbjg2"]
Jul 08 23:04:08.970778 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:04:08.970746    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee533351_5f8b_461e_b284_af522067952f.slice/crio-bf4d578fd4f37337716bee35530e437d38ae0f83426a3965845b45c1e4941268 WatchSource:0}: Error finding container bf4d578fd4f37337716bee35530e437d38ae0f83426a3965845b45c1e4941268: Status 404 returned error can't find the container with id bf4d578fd4f37337716bee35530e437d38ae0f83426a3965845b45c1e4941268
Jul 08 23:04:09.223877 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.223839    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk" event={"ID":"b5a735f9-fd2e-4a5b-8407-0a4f8af80090","Type":"ContainerStarted","Data":"19976b574b1ba02c07af599e7ee7a033c334a645727d7104744c0c7fac3c54bd"}
Jul 08 23:04:09.224062 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.223932    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk" event={"ID":"b5a735f9-fd2e-4a5b-8407-0a4f8af80090","Type":"ContainerStarted","Data":"268d642101db8dce271372ffa7345eccc0dee385a54489beaba1903101982151"}
Jul 08 23:04:09.224062 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.223980    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk" event={"ID":"b5a735f9-fd2e-4a5b-8407-0a4f8af80090","Type":"ContainerStarted","Data":"539a971224b6ad2e868cd0a5524d7f6bccbecc31a287b13e66ac2d5d0b175571"}
Jul 08 23:04:09.225382 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.225355    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2" event={"ID":"ee533351-5f8b-461e-b284-af522067952f","Type":"ContainerStarted","Data":"bf4d578fd4f37337716bee35530e437d38ae0f83426a3965845b45c1e4941268"}
Jul 08 23:04:09.227986 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.227945    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2fss7" event={"ID":"2b56029e-de23-4191-ac78-50d4a48ca94b","Type":"ContainerStarted","Data":"9eb20ed0a9ba7c47675bbc6b04f014585a58401621f8b13dfe7912f524e0b1b3"}
Jul 08 23:04:09.227986 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.227992    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2fss7" event={"ID":"2b56029e-de23-4191-ac78-50d4a48ca94b","Type":"ContainerStarted","Data":"40d702d88a630ae0e414b476647932ae8ffa55d1b7b63b739c38441e3db31c60"}
Jul 08 23:04:09.245679 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.245308    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2fss7" podStartSLOduration=65.306526152 podStartE2EDuration="1m7.245290196s" totalImagesPullingTime="1.938764044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:09.245121602 +0000 UTC m=+68.175155581" watchObservedRunningTime="2026-07-08 23:04:09.245290196 +0000 UTC m=+68.175324171"
Jul 08 23:04:09.341643 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.341563    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:09.444486 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.444397    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:09.677271 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.677231    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:09.683054 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.683016    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 23:04:09.688780 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.688753    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.691279 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.691255    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 08 23:04:09.691573 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.691554    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 08 23:04:09.691664 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.691582    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 08 23:04:09.691664 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.691607    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 08 23:04:09.691765 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.691554    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 08 23:04:09.691884 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.691868    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 08 23:04:09.691942 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.691904    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 08 23:04:09.692219 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.692165    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2j579\""
Jul 08 23:04:09.692424 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.692211    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 08 23:04:09.692587 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.692568    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fpt3gt3gqsucd\""
Jul 08 23:04:09.692824 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.692709    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 08 23:04:09.695317 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.695061    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 08 23:04:09.695317 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.695096    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 08 23:04:09.695317 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.695096    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 08 23:04:09.696109 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.696086    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 08 23:04:09.700682 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.700662    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 08 23:04:09.706073 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.706053    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 23:04:09.792618 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792483    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.792618 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792540    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.792618 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792574    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.792618 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792599    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.792934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792627    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.792934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792671    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.792934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792699    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-web-config\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.792934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792722    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6xg8\" (UniqueName: \"kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-kube-api-access-q6xg8\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.792934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792785    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.792934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792823    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.792934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792852    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.792934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792905    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.793310 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792944    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-config-out\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.793310 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.792984    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-config\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.793310 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.793034    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.793310 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.793081    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.793310 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.793109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.793310 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.793152    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.793310 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.793196    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.793310 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.793232    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.894153 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894112    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-config\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.894153 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894158    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.894395 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894290    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.894395 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894334    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.894395 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894380    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.894584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894407    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.894584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894434    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.894584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894500    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.894584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894534    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.894584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894564    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.894584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894586    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.896779 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894610    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.896779 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894638    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.896779 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894662    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-web-config\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.896779 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894686    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6xg8\" (UniqueName: \"kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-kube-api-access-q6xg8\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.896779 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894713    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.896779 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.894739    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.896779 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.896281    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.903548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.898177    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.903548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.898243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.903548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.898300    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-config-out\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.903548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.898921    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.903548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.899300    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-config\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.903548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.899376    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.903548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.901040    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.903548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.901388    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.903548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.901416    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-config-out\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.903548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.903484    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.906216 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.904583    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.906216 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.904996    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.906216 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.905253    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.906216 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.905363    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.906216 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.905688    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-web-config\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.910924 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.910870    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.911236 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.910944    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.912110 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.912089    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.912420 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.912396    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.912728 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.912706    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.913536 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.913515    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:09.914215 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:09.914192    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6xg8\" (UniqueName: \"kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-kube-api-access-q6xg8\") pod \"prometheus-k8s-0\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:10.001042 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:10.000955    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:10.169261 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:10.169228    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l799w"
Jul 08 23:04:10.353070 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:10.352979    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:10.450199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:10.450159    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:10.550783 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:10.550706    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:10.652555 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:10.652444    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:10.856529 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:10.856488    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:11.209724 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:11.209689    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:11.290792 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:11.290761    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:11.340691 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:11.340656    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:11.441439 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:11.441394    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:11.494337 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:11.494242    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:12.007066 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:12.007039    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:12.127616 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:12.127580    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:12.232515 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:12.232482    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:12.656658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:12.656312    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 23:04:12.948149 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:04:12.948120    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf648348a_69c0_4b92_b933_1c5cd0aaa351.slice/crio-46bb5b70c697f5c8fe09c22404c7719c485b6f46b3ca90e85c8066f90a2e2577 WatchSource:0}: Error finding container 46bb5b70c697f5c8fe09c22404c7719c485b6f46b3ca90e85c8066f90a2e2577: Status 404 returned error can't find the container with id 46bb5b70c697f5c8fe09c22404c7719c485b6f46b3ca90e85c8066f90a2e2577
Jul 08 23:04:13.246198 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.246072    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2" event={"ID":"ee533351-5f8b-461e-b284-af522067952f","Type":"ContainerStarted","Data":"a96ed9fc2a4f70f5c387fc6d4a6799277356d905e8859879d058a41b837f023a"}
Jul 08 23:04:13.249071 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.248639    2560 generic.go:356] "Generic (PLEG): container finished" podID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerID="7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c" exitCode=0
Jul 08 23:04:13.249186 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.249106    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerDied","Data":"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"}
Jul 08 23:04:13.249258 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.249191    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerStarted","Data":"46bb5b70c697f5c8fe09c22404c7719c485b6f46b3ca90e85c8066f90a2e2577"}
Jul 08 23:04:13.252373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.252352    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerStarted","Data":"21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa"}
Jul 08 23:04:13.255563 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.255539    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk" event={"ID":"b5a735f9-fd2e-4a5b-8407-0a4f8af80090","Type":"ContainerStarted","Data":"ac4773de89ca3291d2af3b84068fdcb23d8539524e9ee15f6ac91854f0061f20"}
Jul 08 23:04:13.256984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.256965    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l6w79" event={"ID":"6952cf89-883f-4132-8357-52b4f055486f","Type":"ContainerStarted","Data":"37fd312548685ed8ef69e0af5001fa8e5925ce31232704a40ed9d071376a93d4"}
Jul 08 23:04:13.257303 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.257290    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:04:13.258478 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.258437    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w7pd9" event={"ID":"aa634889-de73-473e-915b-56355ad11639","Type":"ContainerStarted","Data":"07121d1027074b842fabb2027bed19977531cf17f639a8ea0c2d081438ea5ffd"}
Jul 08 23:04:13.262557 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.262521    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2" podStartSLOduration=2.121479702 podStartE2EDuration="6.262509059s" totalImagesPullingTime="4.141029357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:13.260819787 +0000 UTC m=+72.190853760" watchObservedRunningTime="2026-07-08 23:04:13.262509059 +0000 UTC m=+72.192543033"
Jul 08 23:04:13.320282 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.320091    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w7pd9" podStartSLOduration=65.335402486 podStartE2EDuration="1m11.320074883s" totalImagesPullingTime="5.984672397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:13.303423524 +0000 UTC m=+72.233457500" watchObservedRunningTime="2026-07-08 23:04:13.320074883 +0000 UTC m=+72.250108877"
Jul 08 23:04:13.428660 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:13.428633    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:14.268914 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.268863    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerStarted","Data":"20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610"}
Jul 08 23:04:14.269350 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.268978    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerStarted","Data":"5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a"}
Jul 08 23:04:14.269350 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.269007    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerStarted","Data":"9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab"}
Jul 08 23:04:14.269350 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.269031    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerStarted","Data":"166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b"}
Jul 08 23:04:14.269350 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.269062    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerStarted","Data":"db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5"}
Jul 08 23:04:14.272512 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.272411    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk" event={"ID":"b5a735f9-fd2e-4a5b-8407-0a4f8af80090","Type":"ContainerStarted","Data":"f8855b2211e17b00772b670569028a149f68e36fc3c74637eaee0155be0b8df6"}
Jul 08 23:04:14.272512 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.272502    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk" event={"ID":"b5a735f9-fd2e-4a5b-8407-0a4f8af80090","Type":"ContainerStarted","Data":"cc6cc62393f3b21f6afbb1cd1f426ed3461d80f03fe2605060485abdbd9aa11b"}
Jul 08 23:04:14.297026 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.296520    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l6w79" podStartSLOduration=66.652276217 podStartE2EDuration="1m12.296503273s" totalImagesPullingTime="5.644227056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:13.318358305 +0000 UTC m=+72.248392281" watchObservedRunningTime="2026-07-08 23:04:14.296503273 +0000 UTC m=+73.226537249"
Jul 08 23:04:14.297707 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.297638    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.627920203 podStartE2EDuration="10.297626759s" totalImagesPullingTime="6.646083466s" totalInitContainerRuntime="23.62309ms" isStatefulPod=false podCreationTimestamp="2026-07-08 23:04:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:14.294309518 +0000 UTC m=+73.224343497" watchObservedRunningTime="2026-07-08 23:04:14.297626759 +0000 UTC m=+73.227660734"
Jul 08 23:04:14.320191 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.319760    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk" podStartSLOduration=2.435665776 podStartE2EDuration="9.31974124s" totalImagesPullingTime="6.884075464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:04:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:14.318602084 +0000 UTC m=+73.248636058" watchObservedRunningTime="2026-07-08 23:04:14.31974124 +0000 UTC m=+73.249775217"
Jul 08 23:04:14.326104 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.326057    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:14.422684 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.422633    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:14.852180 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.852143    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:14.910422 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:14.910378    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:15.277176 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:15.277146    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:15.282605 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:15.282579    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-c4cd44f8-6krxk"
Jul 08 23:04:16.122601 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:16.122566    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:16.177717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:16.177671    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:16.949305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:16.949270    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:17.001164 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:17.001124    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:17.288132 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:17.288077    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerStarted","Data":"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"}
Jul 08 23:04:17.288237 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:17.288205    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerStarted","Data":"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"}
Jul 08 23:04:17.288272 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:17.288263    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerStarted","Data":"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"}
Jul 08 23:04:18.295119 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:18.295084    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerStarted","Data":"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"}
Jul 08 23:04:18.295569 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:18.295161    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerStarted","Data":"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"}
Jul 08 23:04:18.295569 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:18.295199    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerStarted","Data":"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"}
Jul 08 23:04:18.330197 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:18.330154    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.501224175 podStartE2EDuration="9.330136446s" totalImagesPullingTime="3.796175605s" totalInitContainerRuntime="32.736666ms" isStatefulPod=false podCreationTimestamp="2026-07-08 23:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:04:18.328885689 +0000 UTC m=+77.258919662" watchObservedRunningTime="2026-07-08 23:04:18.330136446 +0000 UTC m=+77.260170424"
Jul 08 23:04:20.001453 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:20.001419    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:04:21.043865 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:21.043834    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:21.146205 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:21.146175    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:21.375582 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:21.375504    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:21.475643 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:21.475617    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:23.349961 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:23.349855    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:23.449802 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:23.449777    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:23.633235 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:23.633167    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:23.682160 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:23.682132    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:23.995090 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:23.995060    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:24.072427 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:24.072402    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:27.271708 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:27.271672    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:27.370380 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:27.370348    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:27.967497 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:27.967426    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e229b86-1eee-4335-9cd1-a3a641ea9d50/init-config-reloader/0.log"
Jul 08 23:04:28.166837 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:28.166802    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e229b86-1eee-4335-9cd1-a3a641ea9d50/alertmanager/0.log"
Jul 08 23:04:28.181553 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:28.181533    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:28.181644 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:28.181580    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:28.367207 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:28.367137    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e229b86-1eee-4335-9cd1-a3a641ea9d50/config-reloader/0.log"
Jul 08 23:04:28.566770 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:28.566747    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e229b86-1eee-4335-9cd1-a3a641ea9d50/kube-rbac-proxy-web/0.log"
Jul 08 23:04:28.767314 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:28.767288    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e229b86-1eee-4335-9cd1-a3a641ea9d50/kube-rbac-proxy/0.log"
Jul 08 23:04:28.966369 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:28.966344    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e229b86-1eee-4335-9cd1-a3a641ea9d50/kube-rbac-proxy-metric/0.log"
Jul 08 23:04:29.167313 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:29.167250    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4e229b86-1eee-4335-9cd1-a3a641ea9d50/prom-label-proxy/0.log"
Jul 08 23:04:30.167019 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:30.166992    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-86b868f65f-pbjg2_ee533351-5f8b-461e-b284-af522067952f/metrics-server/0.log"
Jul 08 23:04:31.765934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:31.765910    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wdw8h_f05ce760-1a5c-45ca-a213-752db93a15ee/init-textfile/0.log"
Jul 08 23:04:31.953007 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:31.952972    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:31.967309 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:31.967289    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wdw8h_f05ce760-1a5c-45ca-a213-752db93a15ee/node-exporter/0.log"
Jul 08 23:04:32.007361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:32.007337    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:32.171716 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:32.171694    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wdw8h_f05ce760-1a5c-45ca-a213-752db93a15ee/kube-rbac-proxy/0.log"
Jul 08 23:04:32.232723 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:32.232692    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:32.333172 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:32.333137    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:32.967199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:32.967129    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f648348a-69c0-4b92-b933-1c5cd0aaa351/init-config-reloader/0.log"
Jul 08 23:04:33.167760 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:33.167727    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f648348a-69c0-4b92-b933-1c5cd0aaa351/prometheus/0.log"
Jul 08 23:04:33.370826 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:33.370763    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f648348a-69c0-4b92-b933-1c5cd0aaa351/config-reloader/0.log"
Jul 08 23:04:33.566975 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:33.566947    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f648348a-69c0-4b92-b933-1c5cd0aaa351/thanos-sidecar/0.log"
Jul 08 23:04:33.767266 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:33.767239    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f648348a-69c0-4b92-b933-1c5cd0aaa351/kube-rbac-proxy-web/0.log"
Jul 08 23:04:33.967459 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:33.967426    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f648348a-69c0-4b92-b933-1c5cd0aaa351/kube-rbac-proxy/0.log"
Jul 08 23:04:34.167558 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:34.167494    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f648348a-69c0-4b92-b933-1c5cd0aaa351/kube-rbac-proxy-thanos/0.log"
Jul 08 23:04:34.367686 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:34.367646    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-lkpl9_7089b439-fd61-4b8a-8cd1-bbf86c74809d/prometheus-operator/0.log"
Jul 08 23:04:34.566609 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:34.566569    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-lkpl9_7089b439-fd61-4b8a-8cd1-bbf86c74809d/kube-rbac-proxy/0.log"
Jul 08 23:04:34.966701 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:34.966670    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/thanos-query/0.log"
Jul 08 23:04:35.167212 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:35.167186    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/kube-rbac-proxy-web/0.log"
Jul 08 23:04:35.366746 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:35.366679    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/kube-rbac-proxy/0.log"
Jul 08 23:04:35.566914 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:35.566886    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/prom-label-proxy/0.log"
Jul 08 23:04:35.766129 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:35.766103    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/kube-rbac-proxy-rules/0.log"
Jul 08 23:04:35.967294 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:35.967264    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/kube-rbac-proxy-metrics/0.log"
Jul 08 23:04:37.767307 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:37.767276    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-smf48_efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb/serve-healthcheck-canary/0.log"
Jul 08 23:04:43.433894 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:43.433861    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:45.230775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:45.230732    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:45.279285 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:45.279261    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l6w79"
Jul 08 23:04:45.283706 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:45.283685    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:46.955832 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:46.955787    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:47.009656 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:47.009627    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:48.186210 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:48.186183    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:48.189892 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:48.189873    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-86b868f65f-pbjg2"
Jul 08 23:04:54.269956 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:54.269927    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:54.370382 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:54.370351    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:04:54.470527 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:04:54.470490    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:02.015235 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:02.015197    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:02.092477 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:02.092441    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:10.001327 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:10.001284    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:10.020184 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:10.020160    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:10.499316 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:10.499288    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:22.650013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:22.649976    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:23.353728 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:23.353692    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:23.709404 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:23.709375    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 23:05:23.709779 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:23.709682    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="alertmanager" containerID="cri-o://21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa" gracePeriod=120
Jul 08 23:05:23.709779 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:23.709760    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy-metric" containerID="cri-o://5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a" gracePeriod=120
Jul 08 23:05:23.709870 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:23.709780    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy-web" containerID="cri-o://166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b" gracePeriod=120
Jul 08 23:05:23.709870 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:23.709806    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="config-reloader" containerID="cri-o://db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5" gracePeriod=120
Jul 08 23:05:23.709870 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:23.709826    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy" containerID="cri-o://9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab" gracePeriod=120
Jul 08 23:05:23.710008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:23.709871    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="prom-label-proxy" containerID="cri-o://20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610" gracePeriod=120
Jul 08 23:05:24.539398 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:24.539363    2560 generic.go:356] "Generic (PLEG): container finished" podID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerID="20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610" exitCode=0
Jul 08 23:05:24.539398 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:24.539387    2560 generic.go:356] "Generic (PLEG): container finished" podID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerID="5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a" exitCode=0
Jul 08 23:05:24.539398 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:24.539394    2560 generic.go:356] "Generic (PLEG): container finished" podID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerID="9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab" exitCode=0
Jul 08 23:05:24.539398 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:24.539399    2560 generic.go:356] "Generic (PLEG): container finished" podID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerID="db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5" exitCode=0
Jul 08 23:05:24.539398 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:24.539404    2560 generic.go:356] "Generic (PLEG): container finished" podID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerID="21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa" exitCode=0
Jul 08 23:05:24.539748 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:24.539441    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerDied","Data":"20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610"}
Jul 08 23:05:24.539748 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:24.539557    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerDied","Data":"5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a"}
Jul 08 23:05:24.539748 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:24.539589    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerDied","Data":"9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab"}
Jul 08 23:05:24.539748 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:24.539613    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerDied","Data":"db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5"}
Jul 08 23:05:24.539748 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:24.539644    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerDied","Data":"21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa"}
Jul 08 23:05:24.941849 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:24.941825    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.080481 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080396    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-main-db\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.080481 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080428    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h64p\" (UniqueName: \"kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-kube-api-access-7h64p\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.080481 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080450    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.080702 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080488    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-tls-assets\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.080702 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080622    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-volume\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.080702 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080665    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-main-tls\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.080702 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080698    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-web-config\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.080896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080733    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-web\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.080896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080768    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:05:25.080896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080781    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-metrics-client-ca\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.080896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080814    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.080896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080883    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-cluster-tls-config\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.081152 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080911    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-trusted-ca-bundle\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.081152 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.080943    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-out\") pod \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\" (UID: \"4e229b86-1eee-4335-9cd1-a3a641ea9d50\") "
Jul 08 23:05:25.081252 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.081197    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-main-db\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.082172 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.081951    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:05:25.082172 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.082127    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:05:25.083390 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.083365    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-kube-api-access-7h64p" (OuterVolumeSpecName: "kube-api-access-7h64p") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "kube-api-access-7h64p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:05:25.083621 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.083602    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-out" (OuterVolumeSpecName: "config-out") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:05:25.084019 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.083992    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:25.084373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.084351    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:25.084680 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.084660    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:05:25.084763 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.084700    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:25.084805 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.084767    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-volume" (OuterVolumeSpecName: "config-volume") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:25.085224 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.085204    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:25.088418 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.088393    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:25.093934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.093908    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-web-config" (OuterVolumeSpecName: "web-config") pod "4e229b86-1eee-4335-9cd1-a3a641ea9d50" (UID: "4e229b86-1eee-4335-9cd1-a3a641ea9d50"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:25.181739 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181715    2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-cluster-tls-config\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.181827 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181742    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.181827 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181753    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-out\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.181827 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181761    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h64p\" (UniqueName: \"kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-kube-api-access-7h64p\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.181827 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181772    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.181827 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181786    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e229b86-1eee-4335-9cd1-a3a641ea9d50-tls-assets\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.181827 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181798    2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-config-volume\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.181827 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181807    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-main-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.181827 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181817    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-web-config\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.182062 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181829    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.182062 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181842    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e229b86-1eee-4335-9cd1-a3a641ea9d50-metrics-client-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.182062 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.181855    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e229b86-1eee-4335-9cd1-a3a641ea9d50-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:25.546217 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.546187    2560 generic.go:356] "Generic (PLEG): container finished" podID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerID="166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b" exitCode=0
Jul 08 23:05:25.546450 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.546232    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerDied","Data":"166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b"}
Jul 08 23:05:25.546450 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.546260    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.546450 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.546284    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e229b86-1eee-4335-9cd1-a3a641ea9d50","Type":"ContainerDied","Data":"ad286a7401fa867a09e164fa3c0f2cb4397986d192ba46a4527359f65aa5ddb0"}
Jul 08 23:05:25.546900 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.546323    2560 scope.go:117] "RemoveContainer" containerID="20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610"
Jul 08 23:05:25.554696 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.554606    2560 scope.go:117] "RemoveContainer" containerID="5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a"
Jul 08 23:05:25.560986 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.560969    2560 scope.go:117] "RemoveContainer" containerID="9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab"
Jul 08 23:05:25.566938 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.566923    2560 scope.go:117] "RemoveContainer" containerID="166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b"
Jul 08 23:05:25.570297 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.570275    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 23:05:25.574545 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.574528    2560 scope.go:117] "RemoveContainer" containerID="db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5"
Jul 08 23:05:25.575495 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.575398    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 23:05:25.581774 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.581759    2560 scope.go:117] "RemoveContainer" containerID="21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa"
Jul 08 23:05:25.588067 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.588055    2560 scope.go:117] "RemoveContainer" containerID="36c56140a9836c5be7a58f673bb2075e6efb52c0fc715600b5c56d6754c0ab3e"
Jul 08 23:05:25.594193 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.594177    2560 scope.go:117] "RemoveContainer" containerID="20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610"
Jul 08 23:05:25.594426 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:25.594408    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610\": container with ID starting with 20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610 not found: ID does not exist" containerID="20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610"
Jul 08 23:05:25.594484 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.594433    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610"} err="failed to get container status \"20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610\": rpc error: code = NotFound desc = could not find container \"20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610\": container with ID starting with 20f545a81f15bfbd670f35dcab50e4e175c4b2135eb19cf2efad549487eab610 not found: ID does not exist"
Jul 08 23:05:25.594484 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.594460    2560 scope.go:117] "RemoveContainer" containerID="5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a"
Jul 08 23:05:25.594697 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:25.594680    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a\": container with ID starting with 5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a not found: ID does not exist" containerID="5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a"
Jul 08 23:05:25.594744 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.594701    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a"} err="failed to get container status \"5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a\": rpc error: code = NotFound desc = could not find container \"5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a\": container with ID starting with 5417db1a843154247bbd8eab61f5c13e94a3455f921b85caa4ea6eaf5b92534a not found: ID does not exist"
Jul 08 23:05:25.594744 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.594713    2560 scope.go:117] "RemoveContainer" containerID="9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab"
Jul 08 23:05:25.594899 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:25.594884    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab\": container with ID starting with 9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab not found: ID does not exist" containerID="9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab"
Jul 08 23:05:25.594937 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.594903    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab"} err="failed to get container status \"9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab\": rpc error: code = NotFound desc = could not find container \"9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab\": container with ID starting with 9b689d6cf069b18ead516a16de9f80b71335a003ef2dc4289230fbe2fc581cab not found: ID does not exist"
Jul 08 23:05:25.594937 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.594915    2560 scope.go:117] "RemoveContainer" containerID="166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b"
Jul 08 23:05:25.595134 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:25.595118    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b\": container with ID starting with 166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b not found: ID does not exist" containerID="166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b"
Jul 08 23:05:25.595171 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.595139    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b"} err="failed to get container status \"166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b\": rpc error: code = NotFound desc = could not find container \"166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b\": container with ID starting with 166ab3d41f75a7b3da298a4c3ce68228ffea2a4c124b335c43795b84bedd8d4b not found: ID does not exist"
Jul 08 23:05:25.595171 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.595155    2560 scope.go:117] "RemoveContainer" containerID="db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5"
Jul 08 23:05:25.595348 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:25.595331    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5\": container with ID starting with db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5 not found: ID does not exist" containerID="db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5"
Jul 08 23:05:25.595386 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.595352    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5"} err="failed to get container status \"db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5\": rpc error: code = NotFound desc = could not find container \"db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5\": container with ID starting with db20259c018224abc8bc8ecea4465f8cb72530e469a258f0ba2f2dfad3471de5 not found: ID does not exist"
Jul 08 23:05:25.595386 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.595363    2560 scope.go:117] "RemoveContainer" containerID="21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa"
Jul 08 23:05:25.595564 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:25.595548    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa\": container with ID starting with 21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa not found: ID does not exist" containerID="21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa"
Jul 08 23:05:25.595615 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.595567    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa"} err="failed to get container status \"21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa\": rpc error: code = NotFound desc = could not find container \"21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa\": container with ID starting with 21b9fd51c190eabf73dc4184d22a21ac3e4938ff1e0ad81fa919c1caec0e49aa not found: ID does not exist"
Jul 08 23:05:25.595615 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.595579    2560 scope.go:117] "RemoveContainer" containerID="36c56140a9836c5be7a58f673bb2075e6efb52c0fc715600b5c56d6754c0ab3e"
Jul 08 23:05:25.595829 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:25.595813    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36c56140a9836c5be7a58f673bb2075e6efb52c0fc715600b5c56d6754c0ab3e\": container with ID starting with 36c56140a9836c5be7a58f673bb2075e6efb52c0fc715600b5c56d6754c0ab3e not found: ID does not exist" containerID="36c56140a9836c5be7a58f673bb2075e6efb52c0fc715600b5c56d6754c0ab3e"
Jul 08 23:05:25.595880 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.595841    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c56140a9836c5be7a58f673bb2075e6efb52c0fc715600b5c56d6754c0ab3e"} err="failed to get container status \"36c56140a9836c5be7a58f673bb2075e6efb52c0fc715600b5c56d6754c0ab3e\": rpc error: code = NotFound desc = could not find container \"36c56140a9836c5be7a58f673bb2075e6efb52c0fc715600b5c56d6754c0ab3e\": container with ID starting with 36c56140a9836c5be7a58f673bb2075e6efb52c0fc715600b5c56d6754c0ab3e not found: ID does not exist"
Jul 08 23:05:25.606773 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.606753    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 23:05:25.607005 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.606994    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="prom-label-proxy"
Jul 08 23:05:25.607053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607006    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="prom-label-proxy"
Jul 08 23:05:25.607053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607015    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy-web"
Jul 08 23:05:25.607053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607020    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy-web"
Jul 08 23:05:25.607053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607032    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="config-reloader"
Jul 08 23:05:25.607053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607037    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="config-reloader"
Jul 08 23:05:25.607053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607043    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy-metric"
Jul 08 23:05:25.607053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607049    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy-metric"
Jul 08 23:05:25.607053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607055    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="alertmanager"
Jul 08 23:05:25.607305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607060    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="alertmanager"
Jul 08 23:05:25.607305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607067    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy"
Jul 08 23:05:25.607305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607072    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy"
Jul 08 23:05:25.607305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607080    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="init-config-reloader"
Jul 08 23:05:25.607305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607085    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="init-config-reloader"
Jul 08 23:05:25.607305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607123    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy-web"
Jul 08 23:05:25.607305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607130    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="prom-label-proxy"
Jul 08 23:05:25.607305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607135    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy"
Jul 08 23:05:25.607305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607141    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="config-reloader"
Jul 08 23:05:25.607305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607147    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="kube-rbac-proxy-metric"
Jul 08 23:05:25.607305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.607151    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" containerName="alertmanager"
Jul 08 23:05:25.610279 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.610263    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.612725 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.612709    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-b4578\""
Jul 08 23:05:25.612811 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.612765    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 08 23:05:25.613182 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.613165    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 08 23:05:25.613261 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.613197    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 08 23:05:25.613316 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.613256    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 08 23:05:25.613366 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.613352    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 08 23:05:25.613499 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.613455    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 08 23:05:25.613499 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.613484    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 08 23:05:25.613773 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.613753    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 08 23:05:25.618832 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.618816    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 08 23:05:25.628625 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.628607    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 23:05:25.685893 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.685870    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/172a1281-af1b-4375-9f13-601ea1421bcc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.685978 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.685902    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.685978 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.685921    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.685978 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.685937    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.685978 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.685956    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.686179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.686002    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-config-volume\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.686179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.686022    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx4wd\" (UniqueName: \"kubernetes.io/projected/172a1281-af1b-4375-9f13-601ea1421bcc-kube-api-access-mx4wd\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.686179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.686048    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/172a1281-af1b-4375-9f13-601ea1421bcc-config-out\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.686179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.686070    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/172a1281-af1b-4375-9f13-601ea1421bcc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.686179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.686106    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-web-config\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.686179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.686122    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/172a1281-af1b-4375-9f13-601ea1421bcc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.686179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.686164    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/172a1281-af1b-4375-9f13-601ea1421bcc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.686383 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.686183    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.703108 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.703088    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e229b86-1eee-4335-9cd1-a3a641ea9d50" path="/var/lib/kubelet/pods/4e229b86-1eee-4335-9cd1-a3a641ea9d50/volumes"
Jul 08 23:05:25.786589 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786562    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786676 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786599    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786676 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786625    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786676 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786676 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786668    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-config-volume\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786693    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx4wd\" (UniqueName: \"kubernetes.io/projected/172a1281-af1b-4375-9f13-601ea1421bcc-kube-api-access-mx4wd\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786718    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/172a1281-af1b-4375-9f13-601ea1421bcc-config-out\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786734    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/172a1281-af1b-4375-9f13-601ea1421bcc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786775    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-web-config\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786792    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/172a1281-af1b-4375-9f13-601ea1421bcc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786812    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/172a1281-af1b-4375-9f13-601ea1421bcc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786836    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.786872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.786864    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/172a1281-af1b-4375-9f13-601ea1421bcc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.787757 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.787297    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/172a1281-af1b-4375-9f13-601ea1421bcc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.787862 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.787838    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/172a1281-af1b-4375-9f13-601ea1421bcc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.787939 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.787869    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/172a1281-af1b-4375-9f13-601ea1421bcc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.789846 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.789823    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.790070 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.790028    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-config-volume\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.790164 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.790075    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-web-config\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.790724 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.790289    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.790724 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.790306    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.790724 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.790289    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/172a1281-af1b-4375-9f13-601ea1421bcc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.790724 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.790415    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.790724 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.790686    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/172a1281-af1b-4375-9f13-601ea1421bcc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.791540 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.791524    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/172a1281-af1b-4375-9f13-601ea1421bcc-config-out\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.797837 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.797801    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx4wd\" (UniqueName: \"kubernetes.io/projected/172a1281-af1b-4375-9f13-601ea1421bcc-kube-api-access-mx4wd\") pod \"alertmanager-main-0\" (UID: \"172a1281-af1b-4375-9f13-601ea1421bcc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:25.919649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:25.919614    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 23:05:26.047270 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:26.047243    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 23:05:26.049096 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:05:26.049042    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod172a1281_af1b_4375_9f13_601ea1421bcc.slice/crio-c0cad12b29579989c8aae7459e03313fd3cad31e56d092a5138df7bc36ee573d WatchSource:0}: Error finding container c0cad12b29579989c8aae7459e03313fd3cad31e56d092a5138df7bc36ee573d: Status 404 returned error can't find the container with id c0cad12b29579989c8aae7459e03313fd3cad31e56d092a5138df7bc36ee573d
Jul 08 23:05:26.551825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:26.551790    2560 generic.go:356] "Generic (PLEG): container finished" podID="172a1281-af1b-4375-9f13-601ea1421bcc" containerID="998f6a9284fa5b2c13f9d845ca9527d6ff660a3e93153f18172b1b91dc7dc3c1" exitCode=0
Jul 08 23:05:26.551961 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:26.551862    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"172a1281-af1b-4375-9f13-601ea1421bcc","Type":"ContainerDied","Data":"998f6a9284fa5b2c13f9d845ca9527d6ff660a3e93153f18172b1b91dc7dc3c1"}
Jul 08 23:05:26.551961 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:26.551950    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"172a1281-af1b-4375-9f13-601ea1421bcc","Type":"ContainerStarted","Data":"c0cad12b29579989c8aae7459e03313fd3cad31e56d092a5138df7bc36ee573d"}
Jul 08 23:05:27.558038 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.558001    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"172a1281-af1b-4375-9f13-601ea1421bcc","Type":"ContainerStarted","Data":"72ad9694c2bcf2f9b1789656c37dcdbb4ec708f7c45933b8b5417e40f19ffcca"}
Jul 08 23:05:27.558432 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.558080    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"172a1281-af1b-4375-9f13-601ea1421bcc","Type":"ContainerStarted","Data":"0816db24b10d8f7a53e83193f79545d421b3d5b0149c05ec4e1b6a263d2c99ef"}
Jul 08 23:05:27.558432 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.558121    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"172a1281-af1b-4375-9f13-601ea1421bcc","Type":"ContainerStarted","Data":"c49c2ac97f6410ca89d4b4232440e4a0b2e23b95d3f4b21399b1925a3fc99092"}
Jul 08 23:05:27.558432 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.558155    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"172a1281-af1b-4375-9f13-601ea1421bcc","Type":"ContainerStarted","Data":"1a0b5ce6d905587ef1c1b118b97261b433b83af32bbce49d9b789c015a1fc642"}
Jul 08 23:05:27.558432 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.558191    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"172a1281-af1b-4375-9f13-601ea1421bcc","Type":"ContainerStarted","Data":"b5ae45b849c4681340f5d6dd48f48f93cf22bda61c0b3c8cabb9dd500a3038c9"}
Jul 08 23:05:27.558432 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.558229    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"172a1281-af1b-4375-9f13-601ea1421bcc","Type":"ContainerStarted","Data":"d8ac827a9ebc379d7955466818859a99bda273f68d75589ff6de431ab8f81bd6"}
Jul 08 23:05:27.583642 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.583596    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.569678667 podStartE2EDuration="2.583581269s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.902602ms" isStatefulPod=false podCreationTimestamp="2026-07-08 23:05:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:05:27.581231839 +0000 UTC m=+146.511265812" watchObservedRunningTime="2026-07-08 23:05:27.583581269 +0000 UTC m=+146.513615308"
Jul 08 23:05:27.997387 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.997358    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 23:05:27.997705 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.997666    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="prometheus" containerID="cri-o://52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9" gracePeriod=600
Jul 08 23:05:27.997705 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.997690    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="thanos-sidecar" containerID="cri-o://56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587" gracePeriod=600
Jul 08 23:05:27.997913 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.997740    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="config-reloader" containerID="cri-o://54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7" gracePeriod=600
Jul 08 23:05:27.997913 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.997784    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy-thanos" containerID="cri-o://789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff" gracePeriod=600
Jul 08 23:05:27.997913 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.997677    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy" containerID="cri-o://45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d" gracePeriod=600
Jul 08 23:05:27.997913 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:27.997719    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy-web" containerID="cri-o://5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5" gracePeriod=600
Jul 08 23:05:28.251916 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.251859    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.409709 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.409677    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-1\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.409897 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.409715    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-kube-rbac-proxy\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.409897 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.409744    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-config\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.409897 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.409803    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-thanos-prometheus-http-client-file\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.409897 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.409840    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-0\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.409897 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.409874    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-config-out\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410143 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.409914    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-kubelet-serving-ca-bundle\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410143 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.409939    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-2\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410143 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.409973    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410143 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410005    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-grpc-tls\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410143 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410038    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-metrics-client-ca\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410143 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410070    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:05:28.410143 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410079    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-serving-certs-ca-bundle\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410143 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410131    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-trusted-ca-bundle\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410544 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410158    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-web-config\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410544 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410188    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-metrics-client-certs\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410544 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410213    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-tls\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410544 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410245    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6xg8\" (UniqueName: \"kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-kube-api-access-q6xg8\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410544 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410279    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-db\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410544 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410304    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410544 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410335    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-tls-assets\") pod \"f648348a-69c0-4b92-b933-1c5cd0aaa351\" (UID: \"f648348a-69c0-4b92-b933-1c5cd0aaa351\") "
Jul 08 23:05:28.410544 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410369    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:05:28.410935 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410613    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.410935 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410633    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.411041 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.410938    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:05:28.411174 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.411149    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:05:28.411293 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.411264    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:05:28.411722 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.411304    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:05:28.411722 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.411591    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:05:28.412495 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.412433    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:05:28.413875 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.413839    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:28.414201 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.414179    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:05:28.414277 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.414255    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-config" (OuterVolumeSpecName: "config") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:28.414358 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.414336    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:28.414458 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.414434    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:28.414572 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.414541    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:28.414658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.414636    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-config-out" (OuterVolumeSpecName: "config-out") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:05:28.414708 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.414639    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:28.414708 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.414671    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-kube-api-access-q6xg8" (OuterVolumeSpecName: "kube-api-access-q6xg8") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "kube-api-access-q6xg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:05:28.414789 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.414701    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:28.414789 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.414747    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:28.424380 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.424355    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-web-config" (OuterVolumeSpecName: "web-config") pod "f648348a-69c0-4b92-b933-1c5cd0aaa351" (UID: "f648348a-69c0-4b92-b933-1c5cd0aaa351"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:05:28.511004 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.510925    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511004 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.510954    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511004 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.510963    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511004 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.510973    2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-grpc-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511004 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.510983    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-configmap-metrics-client-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511004 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.510992    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511004 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511000    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-web-config\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511004 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511009    2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-metrics-client-certs\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511017    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511027    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6xg8\" (UniqueName: \"kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-kube-api-access-q6xg8\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511035    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-db\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511043    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511052    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f648348a-69c0-4b92-b933-1c5cd0aaa351-tls-assets\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511061    2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-secret-kube-rbac-proxy\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511068    2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-config\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511076    2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f648348a-69c0-4b92-b933-1c5cd0aaa351-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511084    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f648348a-69c0-4b92-b933-1c5cd0aaa351-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.511319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.511092    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f648348a-69c0-4b92-b933-1c5cd0aaa351-config-out\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:05:28.568378 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568348    2560 generic.go:356] "Generic (PLEG): container finished" podID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerID="789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff" exitCode=0
Jul 08 23:05:28.568378 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568373    2560 generic.go:356] "Generic (PLEG): container finished" podID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerID="45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d" exitCode=0
Jul 08 23:05:28.568378 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568380    2560 generic.go:356] "Generic (PLEG): container finished" podID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerID="5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5" exitCode=0
Jul 08 23:05:28.568378 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568385    2560 generic.go:356] "Generic (PLEG): container finished" podID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerID="56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587" exitCode=0
Jul 08 23:05:28.568973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568390    2560 generic.go:356] "Generic (PLEG): container finished" podID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerID="54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7" exitCode=0
Jul 08 23:05:28.568973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568395    2560 generic.go:356] "Generic (PLEG): container finished" podID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerID="52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9" exitCode=0
Jul 08 23:05:28.568973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568431    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerDied","Data":"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"}
Jul 08 23:05:28.568973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568444    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.568973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568536    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerDied","Data":"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"}
Jul 08 23:05:28.568973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568565    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerDied","Data":"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"}
Jul 08 23:05:28.568973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568590    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerDied","Data":"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"}
Jul 08 23:05:28.568973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568613    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerDied","Data":"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"}
Jul 08 23:05:28.568973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568602    2560 scope.go:117] "RemoveContainer" containerID="789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"
Jul 08 23:05:28.568973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568636    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerDied","Data":"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"}
Jul 08 23:05:28.568973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.568660    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f648348a-69c0-4b92-b933-1c5cd0aaa351","Type":"ContainerDied","Data":"46bb5b70c697f5c8fe09c22404c7719c485b6f46b3ca90e85c8066f90a2e2577"}
Jul 08 23:05:28.576116 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.576099    2560 scope.go:117] "RemoveContainer" containerID="45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"
Jul 08 23:05:28.582835 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.582817    2560 scope.go:117] "RemoveContainer" containerID="5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"
Jul 08 23:05:28.588927 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.588911    2560 scope.go:117] "RemoveContainer" containerID="56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"
Jul 08 23:05:28.594198 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.594181    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 23:05:28.595439 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.595417    2560 scope.go:117] "RemoveContainer" containerID="54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"
Jul 08 23:05:28.598332 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.598314    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 23:05:28.602191 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.602172    2560 scope.go:117] "RemoveContainer" containerID="52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"
Jul 08 23:05:28.608721 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.608703    2560 scope.go:117] "RemoveContainer" containerID="7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"
Jul 08 23:05:28.614942 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.614921    2560 scope.go:117] "RemoveContainer" containerID="789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"
Jul 08 23:05:28.615162 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:28.615146    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": container with ID starting with 789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff not found: ID does not exist" containerID="789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"
Jul 08 23:05:28.615214 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.615169    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"} err="failed to get container status \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": rpc error: code = NotFound desc = could not find container \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": container with ID starting with 789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff not found: ID does not exist"
Jul 08 23:05:28.615214 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.615186    2560 scope.go:117] "RemoveContainer" containerID="45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"
Jul 08 23:05:28.615384 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:28.615369    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": container with ID starting with 45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d not found: ID does not exist" containerID="45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"
Jul 08 23:05:28.615422 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.615388    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"} err="failed to get container status \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": rpc error: code = NotFound desc = could not find container \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": container with ID starting with 45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d not found: ID does not exist"
Jul 08 23:05:28.615422 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.615402    2560 scope.go:117] "RemoveContainer" containerID="5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"
Jul 08 23:05:28.615616 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:28.615600    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": container with ID starting with 5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5 not found: ID does not exist" containerID="5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"
Jul 08 23:05:28.615654 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.615622    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"} err="failed to get container status \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": rpc error: code = NotFound desc = could not find container \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": container with ID starting with 5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5 not found: ID does not exist"
Jul 08 23:05:28.615654 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.615643    2560 scope.go:117] "RemoveContainer" containerID="56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"
Jul 08 23:05:28.615861 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:28.615846    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": container with ID starting with 56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587 not found: ID does not exist" containerID="56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"
Jul 08 23:05:28.615906 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.615863    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"} err="failed to get container status \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": rpc error: code = NotFound desc = could not find container \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": container with ID starting with 56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587 not found: ID does not exist"
Jul 08 23:05:28.615906 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.615876    2560 scope.go:117] "RemoveContainer" containerID="54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"
Jul 08 23:05:28.616058 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:28.616045    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": container with ID starting with 54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7 not found: ID does not exist" containerID="54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"
Jul 08 23:05:28.616099 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.616061    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"} err="failed to get container status \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": rpc error: code = NotFound desc = could not find container \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": container with ID starting with 54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7 not found: ID does not exist"
Jul 08 23:05:28.616099 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.616072    2560 scope.go:117] "RemoveContainer" containerID="52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"
Jul 08 23:05:28.616289 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:28.616275    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": container with ID starting with 52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9 not found: ID does not exist" containerID="52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"
Jul 08 23:05:28.616327 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.616293    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"} err="failed to get container status \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": rpc error: code = NotFound desc = could not find container \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": container with ID starting with 52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9 not found: ID does not exist"
Jul 08 23:05:28.616327 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.616305    2560 scope.go:117] "RemoveContainer" containerID="7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"
Jul 08 23:05:28.616565 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:05:28.616549    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": container with ID starting with 7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c not found: ID does not exist" containerID="7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"
Jul 08 23:05:28.616621 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.616567    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"} err="failed to get container status \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": rpc error: code = NotFound desc = could not find container \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": container with ID starting with 7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c not found: ID does not exist"
Jul 08 23:05:28.616621 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.616579    2560 scope.go:117] "RemoveContainer" containerID="789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"
Jul 08 23:05:28.616764 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.616749    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"} err="failed to get container status \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": rpc error: code = NotFound desc = could not find container \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": container with ID starting with 789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff not found: ID does not exist"
Jul 08 23:05:28.616805 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.616765    2560 scope.go:117] "RemoveContainer" containerID="45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"
Jul 08 23:05:28.616937 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.616919    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"} err="failed to get container status \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": rpc error: code = NotFound desc = could not find container \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": container with ID starting with 45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d not found: ID does not exist"
Jul 08 23:05:28.616987 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.616940    2560 scope.go:117] "RemoveContainer" containerID="5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"
Jul 08 23:05:28.617106 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.617092    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"} err="failed to get container status \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": rpc error: code = NotFound desc = could not find container \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": container with ID starting with 5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5 not found: ID does not exist"
Jul 08 23:05:28.617146 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.617106    2560 scope.go:117] "RemoveContainer" containerID="56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"
Jul 08 23:05:28.617268 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.617254    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"} err="failed to get container status \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": rpc error: code = NotFound desc = could not find container \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": container with ID starting with 56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587 not found: ID does not exist"
Jul 08 23:05:28.617309 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.617268    2560 scope.go:117] "RemoveContainer" containerID="54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"
Jul 08 23:05:28.617426 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.617411    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"} err="failed to get container status \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": rpc error: code = NotFound desc = could not find container \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": container with ID starting with 54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7 not found: ID does not exist"
Jul 08 23:05:28.617529 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.617425    2560 scope.go:117] "RemoveContainer" containerID="52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"
Jul 08 23:05:28.617657 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.617639    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"} err="failed to get container status \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": rpc error: code = NotFound desc = could not find container \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": container with ID starting with 52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9 not found: ID does not exist"
Jul 08 23:05:28.617657 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.617656    2560 scope.go:117] "RemoveContainer" containerID="7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"
Jul 08 23:05:28.617849 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.617834    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"} err="failed to get container status \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": rpc error: code = NotFound desc = could not find container \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": container with ID starting with 7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c not found: ID does not exist"
Jul 08 23:05:28.617892 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.617850    2560 scope.go:117] "RemoveContainer" containerID="789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"
Jul 08 23:05:28.618041 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.618024    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"} err="failed to get container status \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": rpc error: code = NotFound desc = could not find container \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": container with ID starting with 789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff not found: ID does not exist"
Jul 08 23:05:28.618082 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.618042    2560 scope.go:117] "RemoveContainer" containerID="45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"
Jul 08 23:05:28.618233 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.618214    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"} err="failed to get container status \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": rpc error: code = NotFound desc = could not find container \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": container with ID starting with 45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d not found: ID does not exist"
Jul 08 23:05:28.618305 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.618235    2560 scope.go:117] "RemoveContainer" containerID="5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"
Jul 08 23:05:28.618432 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.618417    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"} err="failed to get container status \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": rpc error: code = NotFound desc = could not find container \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": container with ID starting with 5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5 not found: ID does not exist"
Jul 08 23:05:28.618432 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.618431    2560 scope.go:117] "RemoveContainer" containerID="56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"
Jul 08 23:05:28.618668 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.618632    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"} err="failed to get container status \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": rpc error: code = NotFound desc = could not find container \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": container with ID starting with 56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587 not found: ID does not exist"
Jul 08 23:05:28.618668 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.618646    2560 scope.go:117] "RemoveContainer" containerID="54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"
Jul 08 23:05:28.618844 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.618828    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"} err="failed to get container status \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": rpc error: code = NotFound desc = could not find container \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": container with ID starting with 54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7 not found: ID does not exist"
Jul 08 23:05:28.618844 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.618843    2560 scope.go:117] "RemoveContainer" containerID="52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"
Jul 08 23:05:28.619027 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619011    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"} err="failed to get container status \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": rpc error: code = NotFound desc = could not find container \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": container with ID starting with 52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9 not found: ID does not exist"
Jul 08 23:05:28.619027 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619026    2560 scope.go:117] "RemoveContainer" containerID="7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"
Jul 08 23:05:28.619214 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619197    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"} err="failed to get container status \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": rpc error: code = NotFound desc = could not find container \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": container with ID starting with 7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c not found: ID does not exist"
Jul 08 23:05:28.619257 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619215    2560 scope.go:117] "RemoveContainer" containerID="789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"
Jul 08 23:05:28.619392 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619377    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"} err="failed to get container status \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": rpc error: code = NotFound desc = could not find container \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": container with ID starting with 789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff not found: ID does not exist"
Jul 08 23:05:28.619432 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619393    2560 scope.go:117] "RemoveContainer" containerID="45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"
Jul 08 23:05:28.619619 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619600    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"} err="failed to get container status \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": rpc error: code = NotFound desc = could not find container \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": container with ID starting with 45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d not found: ID does not exist"
Jul 08 23:05:28.619619 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619619    2560 scope.go:117] "RemoveContainer" containerID="5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"
Jul 08 23:05:28.619798 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619781    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"} err="failed to get container status \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": rpc error: code = NotFound desc = could not find container \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": container with ID starting with 5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5 not found: ID does not exist"
Jul 08 23:05:28.619845 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619799    2560 scope.go:117] "RemoveContainer" containerID="56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"
Jul 08 23:05:28.619977 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619962    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"} err="failed to get container status \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": rpc error: code = NotFound desc = could not find container \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": container with ID starting with 56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587 not found: ID does not exist"
Jul 08 23:05:28.620018 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.619976    2560 scope.go:117] "RemoveContainer" containerID="54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"
Jul 08 23:05:28.620228 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.620196    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"} err="failed to get container status \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": rpc error: code = NotFound desc = could not find container \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": container with ID starting with 54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7 not found: ID does not exist"
Jul 08 23:05:28.620228 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.620223    2560 scope.go:117] "RemoveContainer" containerID="52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"
Jul 08 23:05:28.620558 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.620515    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"} err="failed to get container status \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": rpc error: code = NotFound desc = could not find container \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": container with ID starting with 52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9 not found: ID does not exist"
Jul 08 23:05:28.620558 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.620538    2560 scope.go:117] "RemoveContainer" containerID="7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"
Jul 08 23:05:28.620990 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.620969    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"} err="failed to get container status \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": rpc error: code = NotFound desc = could not find container \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": container with ID starting with 7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c not found: ID does not exist"
Jul 08 23:05:28.620990 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.620989    2560 scope.go:117] "RemoveContainer" containerID="789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"
Jul 08 23:05:28.621238 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.621219    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"} err="failed to get container status \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": rpc error: code = NotFound desc = could not find container \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": container with ID starting with 789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff not found: ID does not exist"
Jul 08 23:05:28.621289 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.621238    2560 scope.go:117] "RemoveContainer" containerID="45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"
Jul 08 23:05:28.621450 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.621433    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"} err="failed to get container status \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": rpc error: code = NotFound desc = could not find container \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": container with ID starting with 45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d not found: ID does not exist"
Jul 08 23:05:28.621517 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.621452    2560 scope.go:117] "RemoveContainer" containerID="5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"
Jul 08 23:05:28.621665 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.621651    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"} err="failed to get container status \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": rpc error: code = NotFound desc = could not find container \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": container with ID starting with 5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5 not found: ID does not exist"
Jul 08 23:05:28.621703 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.621667    2560 scope.go:117] "RemoveContainer" containerID="56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"
Jul 08 23:05:28.621877 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.621859    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"} err="failed to get container status \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": rpc error: code = NotFound desc = could not find container \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": container with ID starting with 56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587 not found: ID does not exist"
Jul 08 23:05:28.621927 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.621878    2560 scope.go:117] "RemoveContainer" containerID="54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"
Jul 08 23:05:28.621985 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.621964    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 23:05:28.622110 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622093    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"} err="failed to get container status \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": rpc error: code = NotFound desc = could not find container \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": container with ID starting with 54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7 not found: ID does not exist"
Jul 08 23:05:28.622155 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622110    2560 scope.go:117] "RemoveContainer" containerID="52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"
Jul 08 23:05:28.622275 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622241    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy-web"
Jul 08 23:05:28.622333 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622279    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy-web"
Jul 08 23:05:28.622333 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622291    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="thanos-sidecar"
Jul 08 23:05:28.622333 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622297    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="thanos-sidecar"
Jul 08 23:05:28.622333 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622305    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="prometheus"
Jul 08 23:05:28.622333 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622310    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="prometheus"
Jul 08 23:05:28.622333 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622314    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"} err="failed to get container status \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": rpc error: code = NotFound desc = could not find container \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": container with ID starting with 52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9 not found: ID does not exist"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622344    2560 scope.go:117] "RemoveContainer" containerID="7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622322    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622385    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622394    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy-thanos"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622402    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy-thanos"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622421    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="init-config-reloader"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622427    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="init-config-reloader"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622434    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="config-reloader"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622440    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="config-reloader"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622522    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy-web"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622533    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="config-reloader"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622542    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="prometheus"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622549    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="thanos-sidecar"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622559    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy-thanos"
Jul 08 23:05:28.622562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622565    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" containerName="kube-rbac-proxy"
Jul 08 23:05:28.623018 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622563    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"} err="failed to get container status \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": rpc error: code = NotFound desc = could not find container \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": container with ID starting with 7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c not found: ID does not exist"
Jul 08 23:05:28.623018 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622579    2560 scope.go:117] "RemoveContainer" containerID="789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"
Jul 08 23:05:28.623018 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622753    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff"} err="failed to get container status \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": rpc error: code = NotFound desc = could not find container \"789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff\": container with ID starting with 789b491c7cd4c4681e4552c5e4f8afe4e94046a9bc25a7816b7995dde82d83ff not found: ID does not exist"
Jul 08 23:05:28.623018 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622772    2560 scope.go:117] "RemoveContainer" containerID="45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"
Jul 08 23:05:28.623018 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622967    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d"} err="failed to get container status \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": rpc error: code = NotFound desc = could not find container \"45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d\": container with ID starting with 45b1f772602d1f379a34915b1179fa640fbc55f150c55b87fd06a39e5834858d not found: ID does not exist"
Jul 08 23:05:28.623018 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.622984    2560 scope.go:117] "RemoveContainer" containerID="5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"
Jul 08 23:05:28.623255 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.623212    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5"} err="failed to get container status \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": rpc error: code = NotFound desc = could not find container \"5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5\": container with ID starting with 5ab5c594c322f961514422c1a57d138b2d8c12dc7271ab8bdf3931fee6366fd5 not found: ID does not exist"
Jul 08 23:05:28.623255 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.623227    2560 scope.go:117] "RemoveContainer" containerID="56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"
Jul 08 23:05:28.623437 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.623421    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587"} err="failed to get container status \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": rpc error: code = NotFound desc = could not find container \"56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587\": container with ID starting with 56c732c2dbc26ba4378fb1646258af15a0fa0605b60fb620a2ec6f377a5c2587 not found: ID does not exist"
Jul 08 23:05:28.623519 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.623438    2560 scope.go:117] "RemoveContainer" containerID="54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"
Jul 08 23:05:28.623682 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.623665    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7"} err="failed to get container status \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": rpc error: code = NotFound desc = could not find container \"54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7\": container with ID starting with 54ef79d701ca4cd428b2ea872f8301126e03f3f01d77b05ac406a2be062636f7 not found: ID does not exist"
Jul 08 23:05:28.623735 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.623682    2560 scope.go:117] "RemoveContainer" containerID="52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"
Jul 08 23:05:28.623884 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.623867    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9"} err="failed to get container status \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": rpc error: code = NotFound desc = could not find container \"52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9\": container with ID starting with 52d4e8ffc2f458953ce65117cfc6f669482e513e090967290bc6436a513c6bd9 not found: ID does not exist"
Jul 08 23:05:28.623933 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.623884    2560 scope.go:117] "RemoveContainer" containerID="7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"
Jul 08 23:05:28.624047 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.624032    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c"} err="failed to get container status \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": rpc error: code = NotFound desc = could not find container \"7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c\": container with ID starting with 7c7ebbf4b4d0050e6ca682f2a77f2d12ed7f206e228eb19196f45e843f783b3c not found: ID does not exist"
Jul 08 23:05:28.625654 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.625640    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.628165 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.628146    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 08 23:05:28.628425 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.628354    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 08 23:05:28.628641 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.628450    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 08 23:05:28.628641 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.628460    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 08 23:05:28.628641 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.628496    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 08 23:05:28.628641 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.628532    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2j579\""
Jul 08 23:05:28.628868 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.628807    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 08 23:05:28.629334 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.629314    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 08 23:05:28.629422 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.629338    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 08 23:05:28.629422 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.629392    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 08 23:05:28.629422 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.629314    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fpt3gt3gqsucd\""
Jul 08 23:05:28.629594 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.629437    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 08 23:05:28.629914 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.629884    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 08 23:05:28.629914 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.629902    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 08 23:05:28.631369 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.631349    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 08 23:05:28.634901 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.634885    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 08 23:05:28.640576 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.640558    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 23:05:28.712896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.712869    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/efdb6df6-0799-4f8a-bcf2-d98df1439561-config-out\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713081 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.712919    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/efdb6df6-0799-4f8a-bcf2-d98df1439561-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713081 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.712945    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713081 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.712971    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713081 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713013    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713090    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713114    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-config\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713138    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713408 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713323    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713509 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713572 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713523    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713572 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-web-config\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713689 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713619    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713742 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713679    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713783 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713750    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713783 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713777    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713856 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713811    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk6l4\" (UniqueName: \"kubernetes.io/projected/efdb6df6-0799-4f8a-bcf2-d98df1439561-kube-api-access-bk6l4\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713906 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713849    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713906 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713881    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.713967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.713908    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.814755 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.814681    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.814755 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.814712    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.814755 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.814735    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.814755 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.814750    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.814816    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk6l4\" (UniqueName: \"kubernetes.io/projected/efdb6df6-0799-4f8a-bcf2-d98df1439561-kube-api-access-bk6l4\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.814856    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.814884    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.814907    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.814940    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/efdb6df6-0799-4f8a-bcf2-d98df1439561-config-out\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.814969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/efdb6df6-0799-4f8a-bcf2-d98df1439561-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.814994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.815022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.815046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.815085    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.815093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.815118    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-config\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.815146    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.815186    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.815223    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.815249    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.815288    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-web-config\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.815958 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.815579    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.816310 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.816287    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.817853 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.817737    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/efdb6df6-0799-4f8a-bcf2-d98df1439561-config-out\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.818156 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.817962    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.818156 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.818003    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-web-config\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.818156 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.818025    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.818156 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.818063    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/efdb6df6-0799-4f8a-bcf2-d98df1439561-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.818386 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.818325    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.818606 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.818582    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.818682 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.818639    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.818948 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.818886    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-config\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.818948 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.818920    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.819142 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.819123    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.819425 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.819404    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efdb6df6-0799-4f8a-bcf2-d98df1439561-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.820336 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.820315    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.820413 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.820361    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.820537 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.820522    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.820747 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.820726    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/efdb6df6-0799-4f8a-bcf2-d98df1439561-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.822312 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.822296    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk6l4\" (UniqueName: \"kubernetes.io/projected/efdb6df6-0799-4f8a-bcf2-d98df1439561-kube-api-access-bk6l4\") pod \"prometheus-k8s-0\" (UID: \"efdb6df6-0799-4f8a-bcf2-d98df1439561\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:28.935372 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:28.935349    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:29.064457 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:29.064430    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 23:05:29.068439 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:05:29.068415    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefdb6df6_0799_4f8a_bcf2_d98df1439561.slice/crio-fe5816b374b417c5ea04353ef5d021083cdc390bfd81249f9d857d5708f1ca5e WatchSource:0}: Error finding container fe5816b374b417c5ea04353ef5d021083cdc390bfd81249f9d857d5708f1ca5e: Status 404 returned error can't find the container with id fe5816b374b417c5ea04353ef5d021083cdc390bfd81249f9d857d5708f1ca5e
Jul 08 23:05:29.573800 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:29.573764    2560 generic.go:356] "Generic (PLEG): container finished" podID="efdb6df6-0799-4f8a-bcf2-d98df1439561" containerID="d878a5d589a30cb141ee9fea51fe2a4ad6c4d49ab5dd1422e80a923185f517f9" exitCode=0
Jul 08 23:05:29.574195 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:29.573850    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"efdb6df6-0799-4f8a-bcf2-d98df1439561","Type":"ContainerDied","Data":"d878a5d589a30cb141ee9fea51fe2a4ad6c4d49ab5dd1422e80a923185f517f9"}
Jul 08 23:05:29.574195 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:29.573935    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"efdb6df6-0799-4f8a-bcf2-d98df1439561","Type":"ContainerStarted","Data":"fe5816b374b417c5ea04353ef5d021083cdc390bfd81249f9d857d5708f1ca5e"}
Jul 08 23:05:29.704143 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:29.704117    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f648348a-69c0-4b92-b933-1c5cd0aaa351" path="/var/lib/kubelet/pods/f648348a-69c0-4b92-b933-1c5cd0aaa351/volumes"
Jul 08 23:05:30.583619 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:30.583578    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"efdb6df6-0799-4f8a-bcf2-d98df1439561","Type":"ContainerStarted","Data":"3015bc8f118d0240b721db8d4492fd3110d95448ac107cd4e0e44528ec821662"}
Jul 08 23:05:30.584023 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:30.583660    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"efdb6df6-0799-4f8a-bcf2-d98df1439561","Type":"ContainerStarted","Data":"bd04b0e8d8d8f1e09dc38a4a5b534801845c09a7d8c119cadf89ca86873fab37"}
Jul 08 23:05:30.584023 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:30.583706    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"efdb6df6-0799-4f8a-bcf2-d98df1439561","Type":"ContainerStarted","Data":"0f32ec9be18db1d73f2349dbd62907eb0880b12a5849f8a132fe69cf296cdc45"}
Jul 08 23:05:30.584023 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:30.583749    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"efdb6df6-0799-4f8a-bcf2-d98df1439561","Type":"ContainerStarted","Data":"b94552d7324c16b6873315368d15cd05a147ee5ba860c16b3fa79db8b84dd3d2"}
Jul 08 23:05:30.584023 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:30.583789    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"efdb6df6-0799-4f8a-bcf2-d98df1439561","Type":"ContainerStarted","Data":"60f7211449b2799249ad62711ed4643e1e0097bc127ccda507d9a54023066116"}
Jul 08 23:05:30.584023 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:30.583825    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"efdb6df6-0799-4f8a-bcf2-d98df1439561","Type":"ContainerStarted","Data":"a96e6429f708e32e9a9da27a0fa0a13404d85e85e246a0cad161233b874f7df3"}
Jul 08 23:05:30.613307 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:30.613260    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.593324866 podStartE2EDuration="2.613244638s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.919772ms" isStatefulPod=false podCreationTimestamp="2026-07-08 23:05:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:05:30.611525437 +0000 UTC m=+149.541559411" watchObservedRunningTime="2026-07-08 23:05:30.613244638 +0000 UTC m=+149.543278647"
Jul 08 23:05:32.538417 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:32.538388    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:32.591136 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:32.591104    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:33.935666 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:33.935636    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:05:35.726388 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:35.726358    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:43.770474 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:43.770444    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:43.822065 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:43.822038    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:43.875194 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:43.875167    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:57.280850 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:57.280769    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:05:57.381726 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:05:57.381696    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:28.936176 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:28.936144    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:06:28.951124 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:28.951101    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:06:29.811607 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:29.811584    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 23:06:37.902816 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:37.902774    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:37.951236 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:37.951209    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:43.250660 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:43.250628    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:43.304861 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:43.304833    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:43.397844 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:43.397816    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:44.002805 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:44.002775    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:44.104544 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:44.104515    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:49.888962 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:49.888929    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:49.955217 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:49.955186    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:54.051649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:54.051618    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:54.152701 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:54.152671    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:06:54.201849 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:06:54.201810    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:07:01.969105 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:07:01.969055    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:07:02.068972 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:07:02.068946    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:07:19.556788 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:07:19.556752    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:07:36.169953 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:07:36.169921    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:07:37.729008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:07:37.728978    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:01.557760 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:01.557731    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:08:01.562506 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:01.562486    2560 kubelet.go:1648] "Image garbage collection succeeded"
Jul 08 23:08:01.572397 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:01.572380    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:08:02.745276 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:02.745209    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:03.512455 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:03.512425    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:06.357803 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:06.357761    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:06.459118 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:06.459088    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:07.522222 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:07.522188    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:07.625307 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:07.625279    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:07.817862 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:07.817791    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:07.904644 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:07.904615    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:10.170758 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:10.170715    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:10.586796 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:10.586719    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:11.541868 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.541840    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8xk69"]
Jul 08 23:08:11.544714 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.544699    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8xk69"
Jul 08 23:08:11.546944 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.546926    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 08 23:08:11.547899 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.547885    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 08 23:08:11.547959 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.547900    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ncftz\""
Jul 08 23:08:11.554454 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.554435    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8xk69"]
Jul 08 23:08:11.637526 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.637500    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrjm8\" (UniqueName: \"kubernetes.io/projected/06ec2f44-b3c9-4edc-9c53-310fb27e8f3d-kube-api-access-mrjm8\") pod \"cert-manager-cainjector-559cdd8c87-8xk69\" (UID: \"06ec2f44-b3c9-4edc-9c53-310fb27e8f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8xk69"
Jul 08 23:08:11.637632 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.637530    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06ec2f44-b3c9-4edc-9c53-310fb27e8f3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8xk69\" (UID: \"06ec2f44-b3c9-4edc-9c53-310fb27e8f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8xk69"
Jul 08 23:08:11.738438 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.738414    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrjm8\" (UniqueName: \"kubernetes.io/projected/06ec2f44-b3c9-4edc-9c53-310fb27e8f3d-kube-api-access-mrjm8\") pod \"cert-manager-cainjector-559cdd8c87-8xk69\" (UID: \"06ec2f44-b3c9-4edc-9c53-310fb27e8f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8xk69"
Jul 08 23:08:11.738560 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.738442    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06ec2f44-b3c9-4edc-9c53-310fb27e8f3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8xk69\" (UID: \"06ec2f44-b3c9-4edc-9c53-310fb27e8f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8xk69"
Jul 08 23:08:11.746117 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.746098    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06ec2f44-b3c9-4edc-9c53-310fb27e8f3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8xk69\" (UID: \"06ec2f44-b3c9-4edc-9c53-310fb27e8f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8xk69"
Jul 08 23:08:11.746261 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.746245    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrjm8\" (UniqueName: \"kubernetes.io/projected/06ec2f44-b3c9-4edc-9c53-310fb27e8f3d-kube-api-access-mrjm8\") pod \"cert-manager-cainjector-559cdd8c87-8xk69\" (UID: \"06ec2f44-b3c9-4edc-9c53-310fb27e8f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8xk69"
Jul 08 23:08:11.868615 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.868542    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8xk69"
Jul 08 23:08:11.988510 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.988486    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8xk69"]
Jul 08 23:08:11.990323 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:08:11.990299    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06ec2f44_b3c9_4edc_9c53_310fb27e8f3d.slice/crio-da8da8fcd37a1f41477466d9bc42eb3b2040807ec9f633a88b5cedabd60c78eb WatchSource:0}: Error finding container da8da8fcd37a1f41477466d9bc42eb3b2040807ec9f633a88b5cedabd60c78eb: Status 404 returned error can't find the container with id da8da8fcd37a1f41477466d9bc42eb3b2040807ec9f633a88b5cedabd60c78eb
Jul 08 23:08:11.992039 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:11.992025    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 23:08:12.150200 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:12.150120    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8xk69" event={"ID":"06ec2f44-b3c9-4edc-9c53-310fb27e8f3d","Type":"ContainerStarted","Data":"da8da8fcd37a1f41477466d9bc42eb3b2040807ec9f633a88b5cedabd60c78eb"}
Jul 08 23:08:13.158878 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:13.158837    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:13.219168 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:13.219142    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:17.380043 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:17.380006    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:18.174270 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:18.174237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8xk69" event={"ID":"06ec2f44-b3c9-4edc-9c53-310fb27e8f3d","Type":"ContainerStarted","Data":"d9f21903f6709b1407baa9f8450e3fca7ee51c434c51d42a7278c52884f58938"}
Jul 08 23:08:18.189647 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:18.189606    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8xk69" podStartSLOduration=1.510753666 podStartE2EDuration="7.18959215s" totalImagesPullingTime="5.678838484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:08:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:08:18.188591822 +0000 UTC m=+317.118625798" watchObservedRunningTime="2026-07-08 23:08:18.18959215 +0000 UTC m=+317.119626122"
Jul 08 23:08:18.445785 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:18.445679    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:18.501389 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:18.501343    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:20.443445 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:20.443414    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:20.537383 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:20.537331    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:26.033895 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:26.033858    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:26.119892 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:26.119862    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:28.328515 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:28.328485    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:40.509775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.509734    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:40.585525 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.585460    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"]
Jul 08 23:08:40.588298 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.588278    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.594678 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.594275    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 08 23:08:40.594678 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.594365    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 08 23:08:40.595453 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.595428    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 08 23:08:40.595598 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.595452    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 08 23:08:40.595598 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.595523    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 08 23:08:40.595688 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.595643    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-r6445\""
Jul 08 23:08:40.614312 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.614288    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"]
Jul 08 23:08:40.620284 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.620262    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:40.668306 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.668277    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10888b06-ca90-4856-99b0-6c80cea2889d-cert\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.668484 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.668329    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/10888b06-ca90-4856-99b0-6c80cea2889d-metrics-cert\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.668484 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.668378    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/10888b06-ca90-4856-99b0-6c80cea2889d-manager-config\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.668484 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.668411    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpbrm\" (UniqueName: \"kubernetes.io/projected/10888b06-ca90-4856-99b0-6c80cea2889d-kube-api-access-qpbrm\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.769254 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.769163    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/10888b06-ca90-4856-99b0-6c80cea2889d-manager-config\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.769254 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.769213    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpbrm\" (UniqueName: \"kubernetes.io/projected/10888b06-ca90-4856-99b0-6c80cea2889d-kube-api-access-qpbrm\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.769254 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.769261    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10888b06-ca90-4856-99b0-6c80cea2889d-cert\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.769453 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.769296    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/10888b06-ca90-4856-99b0-6c80cea2889d-metrics-cert\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.769815 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.769795    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/10888b06-ca90-4856-99b0-6c80cea2889d-manager-config\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.771674 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.771656    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/10888b06-ca90-4856-99b0-6c80cea2889d-metrics-cert\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.771805 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.771788    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10888b06-ca90-4856-99b0-6c80cea2889d-cert\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.776962 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.776934    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpbrm\" (UniqueName: \"kubernetes.io/projected/10888b06-ca90-4856-99b0-6c80cea2889d-kube-api-access-qpbrm\") pod \"lws-controller-manager-65f9849997-8hwcl\" (UID: \"10888b06-ca90-4856-99b0-6c80cea2889d\") " pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:40.897373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:40.897338    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:41.032393 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:41.032315    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"]
Jul 08 23:08:41.035207 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:08:41.035182    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10888b06_ca90_4856_99b0_6c80cea2889d.slice/crio-61fee9e256c7c7b5e124c761274b5e281d4d1afbf02d73f50dc4a351473924c1 WatchSource:0}: Error finding container 61fee9e256c7c7b5e124c761274b5e281d4d1afbf02d73f50dc4a351473924c1: Status 404 returned error can't find the container with id 61fee9e256c7c7b5e124c761274b5e281d4d1afbf02d73f50dc4a351473924c1
Jul 08 23:08:41.257570 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:41.257532    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl" event={"ID":"10888b06-ca90-4856-99b0-6c80cea2889d","Type":"ContainerStarted","Data":"61fee9e256c7c7b5e124c761274b5e281d4d1afbf02d73f50dc4a351473924c1"}
Jul 08 23:08:44.270145 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:44.270118    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl" event={"ID":"10888b06-ca90-4856-99b0-6c80cea2889d","Type":"ContainerStarted","Data":"315ca5fd7e7847fff8c542c862adf509310dc355fbe98d128aaa987f59be8d7f"}
Jul 08 23:08:44.270537 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:44.270243    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:44.288847 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:44.288805    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl" podStartSLOduration=2.025523691 podStartE2EDuration="4.288790624s" totalImagesPullingTime="2.263266933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:08:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:08:44.287148642 +0000 UTC m=+343.217182620" watchObservedRunningTime="2026-07-08 23:08:44.288790624 +0000 UTC m=+343.218824596"
Jul 08 23:08:44.606207 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:44.606132    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:44.702076 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:44.702045    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:45.214637 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.214604    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:45.392116 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.392088    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:45.503302 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.503224    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:45.552666 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.552640    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"]
Jul 08 23:08:45.555832 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.555818    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.558704 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.558686    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9g6bh\""
Jul 08 23:08:45.558970 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.558956    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 08 23:08:45.559291 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.559279    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 08 23:08:45.563659 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.563644    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 08 23:08:45.567256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.567242    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 08 23:08:45.580935 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.580902    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"]
Jul 08 23:08:45.711666 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.711638    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff97da5f-6b49-418e-b869-8505b2378fe9-webhook-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.711833 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.711690    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff97da5f-6b49-418e-b869-8505b2378fe9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.711833 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.711751    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxzmn\" (UniqueName: \"kubernetes.io/projected/ff97da5f-6b49-418e-b869-8505b2378fe9-kube-api-access-gxzmn\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.711833 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.711782    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ff97da5f-6b49-418e-b869-8505b2378fe9-manifests\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.711833 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.711804    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff97da5f-6b49-418e-b869-8505b2378fe9-tmp\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.742842 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.742816    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:45.812841 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.812754    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff97da5f-6b49-418e-b869-8505b2378fe9-webhook-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.812841 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.812826    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff97da5f-6b49-418e-b869-8505b2378fe9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.813005 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.812865    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxzmn\" (UniqueName: \"kubernetes.io/projected/ff97da5f-6b49-418e-b869-8505b2378fe9-kube-api-access-gxzmn\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.813005 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.812900    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ff97da5f-6b49-418e-b869-8505b2378fe9-manifests\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.813005 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.812936    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff97da5f-6b49-418e-b869-8505b2378fe9-tmp\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.813372 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.813344    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ff97da5f-6b49-418e-b869-8505b2378fe9-manifests\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.815238 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.815216    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff97da5f-6b49-418e-b869-8505b2378fe9-tmp\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.815333 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.815280    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff97da5f-6b49-418e-b869-8505b2378fe9-webhook-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.815425 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.815409    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff97da5f-6b49-418e-b869-8505b2378fe9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.822174 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.822151    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxzmn\" (UniqueName: \"kubernetes.io/projected/ff97da5f-6b49-418e-b869-8505b2378fe9-kube-api-access-gxzmn\") pod \"opendatahub-operator-controller-manager-ccc7894f9-p2gp6\" (UID: \"ff97da5f-6b49-418e-b869-8505b2378fe9\") " pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.841864 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.841839    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:45.865357 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.865331    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:45.947805 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.947761    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:45.996687 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:45.996664    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"]
Jul 08 23:08:45.999163 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:08:45.999131    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff97da5f_6b49_418e_b869_8505b2378fe9.slice/crio-78d305b8fe6b66e1ec889d8f265909a82808b0baf0102d451e4b06e2fefb38e3 WatchSource:0}: Error finding container 78d305b8fe6b66e1ec889d8f265909a82808b0baf0102d451e4b06e2fefb38e3: Status 404 returned error can't find the container with id 78d305b8fe6b66e1ec889d8f265909a82808b0baf0102d451e4b06e2fefb38e3
Jul 08 23:08:46.012461 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:46.012432    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:46.066680 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:46.066603    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:46.164892 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:46.164864    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:46.266390 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:46.266364    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:46.279486 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:46.279443    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6" event={"ID":"ff97da5f-6b49-418e-b869-8505b2378fe9","Type":"ContainerStarted","Data":"78d305b8fe6b66e1ec889d8f265909a82808b0baf0102d451e4b06e2fefb38e3"}
Jul 08 23:08:50.299047 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:50.299012    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6" event={"ID":"ff97da5f-6b49-418e-b869-8505b2378fe9","Type":"ContainerStarted","Data":"f42fec067dc9a0549e3e301664d6466c57362e103e6784e21104c35b813a7f51"}
Jul 08 23:08:51.305546 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:51.305513    2560 generic.go:356] "Generic (PLEG): container finished" podID="ff97da5f-6b49-418e-b869-8505b2378fe9" containerID="f42fec067dc9a0549e3e301664d6466c57362e103e6784e21104c35b813a7f51" exitCode=0
Jul 08 23:08:51.305930 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:51.305594    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6" event={"ID":"ff97da5f-6b49-418e-b869-8505b2378fe9","Type":"ContainerDied","Data":"f42fec067dc9a0549e3e301664d6466c57362e103e6784e21104c35b813a7f51"}
Jul 08 23:08:52.311480 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:52.311431    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6" event={"ID":"ff97da5f-6b49-418e-b869-8505b2378fe9","Type":"ContainerStarted","Data":"8639a0b5ac194d297a183a51c5ed1907e14992e262933a750432a2b75c31322e"}
Jul 08 23:08:52.311892 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:52.311606    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:08:52.333153 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:52.333102    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6" podStartSLOduration=2.582705748 podStartE2EDuration="7.333089072s" totalImagesPullingTime="3.723108134s" totalInitContainerRuntime="1.02727519s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:08:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:08:52.33155988 +0000 UTC m=+351.261593855" watchObservedRunningTime="2026-07-08 23:08:52.333089072 +0000 UTC m=+351.263123044"
Jul 08 23:08:55.277366 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:55.277291    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65f9849997-8hwcl"
Jul 08 23:08:57.407960 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:57.407922    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:08:57.511843 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:08:57.511816    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:01.610645 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:01.610606    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:02.040483 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:02.040409    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:02.144645 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:02.144614    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:03.317540 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:03.317512    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-ccc7894f9-p2gp6"
Jul 08 23:09:04.807439 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:04.807411    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:04.926377 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:04.926348    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:05.628910 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.628880    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-l97bd"]
Jul 08 23:09:05.637142 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.637122    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-l97bd"
Jul 08 23:09:05.639688 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.639663    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 08 23:09:05.639803 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.639746    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tldh2\""
Jul 08 23:09:05.648540 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.648520    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-l97bd"]
Jul 08 23:09:05.672696 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.672673    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:05.673703 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.673685    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9scpp\" (UniqueName: \"kubernetes.io/projected/2980ea4f-2586-4c63-b48a-ecdd785b07cb-kube-api-access-9scpp\") pod \"odh-model-controller-7998c67dc4-l97bd\" (UID: \"2980ea4f-2586-4c63-b48a-ecdd785b07cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-l97bd"
Jul 08 23:09:05.673775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.673713    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2980ea4f-2586-4c63-b48a-ecdd785b07cb-cert\") pod \"odh-model-controller-7998c67dc4-l97bd\" (UID: \"2980ea4f-2586-4c63-b48a-ecdd785b07cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-l97bd"
Jul 08 23:09:05.743639 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.743611    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:05.774627 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.774599    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9scpp\" (UniqueName: \"kubernetes.io/projected/2980ea4f-2586-4c63-b48a-ecdd785b07cb-kube-api-access-9scpp\") pod \"odh-model-controller-7998c67dc4-l97bd\" (UID: \"2980ea4f-2586-4c63-b48a-ecdd785b07cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-l97bd"
Jul 08 23:09:05.774758 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.774630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2980ea4f-2586-4c63-b48a-ecdd785b07cb-cert\") pod \"odh-model-controller-7998c67dc4-l97bd\" (UID: \"2980ea4f-2586-4c63-b48a-ecdd785b07cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-l97bd"
Jul 08 23:09:05.774758 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:09:05.774739    2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 08 23:09:05.774874 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:09:05.774791    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2980ea4f-2586-4c63-b48a-ecdd785b07cb-cert podName:2980ea4f-2586-4c63-b48a-ecdd785b07cb nodeName:}" failed. No retries permitted until 2026-07-08 23:09:06.274774675 +0000 UTC m=+365.204808628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2980ea4f-2586-4c63-b48a-ecdd785b07cb-cert") pod "odh-model-controller-7998c67dc4-l97bd" (UID: "2980ea4f-2586-4c63-b48a-ecdd785b07cb") : secret "odh-model-controller-webhook-cert" not found
Jul 08 23:09:05.782709 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.782687    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9scpp\" (UniqueName: \"kubernetes.io/projected/2980ea4f-2586-4c63-b48a-ecdd785b07cb-kube-api-access-9scpp\") pod \"odh-model-controller-7998c67dc4-l97bd\" (UID: \"2980ea4f-2586-4c63-b48a-ecdd785b07cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-l97bd"
Jul 08 23:09:05.843547 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.843513    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:05.962334 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:05.962312    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:06.045279 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:06.045253    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:06.254809 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:06.254740    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:06.278859 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:06.278780    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2980ea4f-2586-4c63-b48a-ecdd785b07cb-cert\") pod \"odh-model-controller-7998c67dc4-l97bd\" (UID: \"2980ea4f-2586-4c63-b48a-ecdd785b07cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-l97bd"
Jul 08 23:09:06.281079 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:06.281055    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2980ea4f-2586-4c63-b48a-ecdd785b07cb-cert\") pod \"odh-model-controller-7998c67dc4-l97bd\" (UID: \"2980ea4f-2586-4c63-b48a-ecdd785b07cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-l97bd"
Jul 08 23:09:06.349976 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:06.349954    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:06.547022 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:06.546926    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-l97bd"
Jul 08 23:09:06.670936 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:06.670911    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-l97bd"]
Jul 08 23:09:06.672830 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:09:06.672799    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2980ea4f_2586_4c63_b48a_ecdd785b07cb.slice/crio-946eb9f1fb8418d4ef925f15a27a208da604a29fb392cc34648d5d5fb5faa85a WatchSource:0}: Error finding container 946eb9f1fb8418d4ef925f15a27a208da604a29fb392cc34648d5d5fb5faa85a: Status 404 returned error can't find the container with id 946eb9f1fb8418d4ef925f15a27a208da604a29fb392cc34648d5d5fb5faa85a
Jul 08 23:09:07.370590 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:07.370552    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-l97bd" event={"ID":"2980ea4f-2586-4c63-b48a-ecdd785b07cb","Type":"ContainerStarted","Data":"946eb9f1fb8418d4ef925f15a27a208da604a29fb392cc34648d5d5fb5faa85a"}
Jul 08 23:09:10.385449 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:10.385410    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-l97bd" event={"ID":"2980ea4f-2586-4c63-b48a-ecdd785b07cb","Type":"ContainerStarted","Data":"25d7515e4d0633f6262cdd9977b34cbfc1d11c73f57a6cd8d03d06f75b3383fc"}
Jul 08 23:09:10.385851 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:10.385534    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-l97bd"
Jul 08 23:09:10.401334 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:10.401294    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-l97bd" podStartSLOduration=2.225797104 podStartE2EDuration="5.401279849s" totalImagesPullingTime="3.175482745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:09:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:09:10.400054097 +0000 UTC m=+369.330088072" watchObservedRunningTime="2026-07-08 23:09:10.401279849 +0000 UTC m=+369.331313863"
Jul 08 23:09:14.614151 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:14.614114    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:14.712363 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:14.712320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:14.816687 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:14.816657    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:14.921312 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:14.921280    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:15.932326 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:15.932282    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:16.023685 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.023652    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:16.123731 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.123697    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:16.188880 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.188808    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"]
Jul 08 23:09:16.196552 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.196531    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"
Jul 08 23:09:16.199645 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.199624    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 08 23:09:16.200378 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.200355    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 08 23:09:16.200516 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.200380    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 08 23:09:16.200516 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.200394    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 08 23:09:16.201276 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.201262    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-n5ms9\""
Jul 08 23:09:16.205423 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.205403    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"]
Jul 08 23:09:16.253408 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.253378    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:16.260603 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.260579    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/313ba2f0-c401-44e6-a1b4-7e191541c081-tls-certs\") pod \"kube-auth-proxy-7dc6c875b-g2ck2\" (UID: \"313ba2f0-c401-44e6-a1b4-7e191541c081\") " pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"
Jul 08 23:09:16.260714 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.260621    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/313ba2f0-c401-44e6-a1b4-7e191541c081-tmp\") pod \"kube-auth-proxy-7dc6c875b-g2ck2\" (UID: \"313ba2f0-c401-44e6-a1b4-7e191541c081\") " pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"
Jul 08 23:09:16.260714 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.260648    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f66wj\" (UniqueName: \"kubernetes.io/projected/313ba2f0-c401-44e6-a1b4-7e191541c081-kube-api-access-f66wj\") pod \"kube-auth-proxy-7dc6c875b-g2ck2\" (UID: \"313ba2f0-c401-44e6-a1b4-7e191541c081\") " pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"
Jul 08 23:09:16.361484 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.361440    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f66wj\" (UniqueName: \"kubernetes.io/projected/313ba2f0-c401-44e6-a1b4-7e191541c081-kube-api-access-f66wj\") pod \"kube-auth-proxy-7dc6c875b-g2ck2\" (UID: \"313ba2f0-c401-44e6-a1b4-7e191541c081\") " pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"
Jul 08 23:09:16.361645 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.361520    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/313ba2f0-c401-44e6-a1b4-7e191541c081-tls-certs\") pod \"kube-auth-proxy-7dc6c875b-g2ck2\" (UID: \"313ba2f0-c401-44e6-a1b4-7e191541c081\") " pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"
Jul 08 23:09:16.361645 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.361558    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/313ba2f0-c401-44e6-a1b4-7e191541c081-tmp\") pod \"kube-auth-proxy-7dc6c875b-g2ck2\" (UID: \"313ba2f0-c401-44e6-a1b4-7e191541c081\") " pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"
Jul 08 23:09:16.363756 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.363733    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/313ba2f0-c401-44e6-a1b4-7e191541c081-tmp\") pod \"kube-auth-proxy-7dc6c875b-g2ck2\" (UID: \"313ba2f0-c401-44e6-a1b4-7e191541c081\") " pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"
Jul 08 23:09:16.363976 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.363959    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/313ba2f0-c401-44e6-a1b4-7e191541c081-tls-certs\") pod \"kube-auth-proxy-7dc6c875b-g2ck2\" (UID: \"313ba2f0-c401-44e6-a1b4-7e191541c081\") " pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"
Jul 08 23:09:16.373288 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.373262    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f66wj\" (UniqueName: \"kubernetes.io/projected/313ba2f0-c401-44e6-a1b4-7e191541c081-kube-api-access-f66wj\") pod \"kube-auth-proxy-7dc6c875b-g2ck2\" (UID: \"313ba2f0-c401-44e6-a1b4-7e191541c081\") " pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"
Jul 08 23:09:16.394683 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.394659    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:16.506639 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.506541    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"
Jul 08 23:09:16.637610 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:16.637578    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2"]
Jul 08 23:09:16.640776 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:09:16.640751    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod313ba2f0_c401_44e6_a1b4_7e191541c081.slice/crio-cb0c3dc10162fac41af5e866f9e2196c04908bcd11496feab7df56b8248cd56e WatchSource:0}: Error finding container cb0c3dc10162fac41af5e866f9e2196c04908bcd11496feab7df56b8248cd56e: Status 404 returned error can't find the container with id cb0c3dc10162fac41af5e866f9e2196c04908bcd11496feab7df56b8248cd56e
Jul 08 23:09:17.415570 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:17.415509    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2" event={"ID":"313ba2f0-c401-44e6-a1b4-7e191541c081","Type":"ContainerStarted","Data":"cb0c3dc10162fac41af5e866f9e2196c04908bcd11496feab7df56b8248cd56e"}
Jul 08 23:09:17.851900 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:17.851869    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:17.952086 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:17.952056    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:19.074852 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:19.074823    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:20.429065 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:20.429039    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2" event={"ID":"313ba2f0-c401-44e6-a1b4-7e191541c081","Type":"ContainerStarted","Data":"f0eab7a4c828652fd3488ecc4806d12159d0561f5da1934f11d9e16b8f89f839"}
Jul 08 23:09:20.451817 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:20.447434    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7dc6c875b-g2ck2" podStartSLOduration=1.256087218 podStartE2EDuration="4.447414499s" totalImagesPullingTime="3.191327281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:09:20.445845236 +0000 UTC m=+379.375879208" watchObservedRunningTime="2026-07-08 23:09:20.447414499 +0000 UTC m=+379.377448474"
Jul 08 23:09:20.639086 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:20.639040    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:20.696406 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:20.696341    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:21.392217 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:21.392194    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-l97bd"
Jul 08 23:09:22.626386 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:22.626351    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:22.725735 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:22.725707    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:22.900714 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:22.900647    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:23.067481 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:23.067426    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:09:23.199445 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:09:23.199414    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:06.267553 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:06.267509    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:06.368155 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:06.368106    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:06.464508 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:06.464476    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:07.176657 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:07.176624    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:07.375739 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:07.375710    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:11.549669 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:11.549636    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:13.496006 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:13.495971    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:34.938988 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:34.938948    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:35.540587 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:35.540550    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:40.083868 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:40.083841    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:40.175075 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:40.175044    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:40.281626 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:40.281594    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:40.381229 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:40.381144    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:40.768449 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:40.768415    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:40.867008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:40.866976    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:42.387019 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:42.386980    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:43.361443 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:43.361411    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:44.190517 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:44.190488    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:45.715789 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:45.715758    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv"]
Jul 08 23:10:45.718946 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:45.718930    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv"
Jul 08 23:10:45.723002 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:45.722979    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wzdq2\""
Jul 08 23:10:45.723113 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:45.723042    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 08 23:10:45.723845 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:45.723824    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 08 23:10:45.723932 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:45.723887    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 08 23:10:45.732244 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:45.732225    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv"]
Jul 08 23:10:45.810105 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:45.810068    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsrx5\" (UniqueName: \"kubernetes.io/projected/67b264e3-af9e-4284-9ba7-a94656ba2ad9-kube-api-access-jsrx5\") pod \"dns-operator-controller-manager-864dd78c4c-249lv\" (UID: \"67b264e3-af9e-4284-9ba7-a94656ba2ad9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv"
Jul 08 23:10:45.910521 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:45.910482    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsrx5\" (UniqueName: \"kubernetes.io/projected/67b264e3-af9e-4284-9ba7-a94656ba2ad9-kube-api-access-jsrx5\") pod \"dns-operator-controller-manager-864dd78c4c-249lv\" (UID: \"67b264e3-af9e-4284-9ba7-a94656ba2ad9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv"
Jul 08 23:10:45.922110 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:45.922084    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsrx5\" (UniqueName: \"kubernetes.io/projected/67b264e3-af9e-4284-9ba7-a94656ba2ad9-kube-api-access-jsrx5\") pod \"dns-operator-controller-manager-864dd78c4c-249lv\" (UID: \"67b264e3-af9e-4284-9ba7-a94656ba2ad9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv"
Jul 08 23:10:46.029089 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:46.028997    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv"
Jul 08 23:10:46.155503 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:46.155375    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv"]
Jul 08 23:10:46.158426 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:10:46.158385    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b264e3_af9e_4284_9ba7_a94656ba2ad9.slice/crio-9d45f8fb65894456b0c2bb89199f6069e0f5e58fa3be420a53d8f13666a53c21 WatchSource:0}: Error finding container 9d45f8fb65894456b0c2bb89199f6069e0f5e58fa3be420a53d8f13666a53c21: Status 404 returned error can't find the container with id 9d45f8fb65894456b0c2bb89199f6069e0f5e58fa3be420a53d8f13666a53c21
Jul 08 23:10:46.356109 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:46.356026    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:46.759043 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:46.759011    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv" event={"ID":"67b264e3-af9e-4284-9ba7-a94656ba2ad9","Type":"ContainerStarted","Data":"9d45f8fb65894456b0c2bb89199f6069e0f5e58fa3be420a53d8f13666a53c21"}
Jul 08 23:10:46.806372 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:46.806338    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:48.773087 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:48.773063    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv" event={"ID":"67b264e3-af9e-4284-9ba7-a94656ba2ad9","Type":"ContainerStarted","Data":"a25fad51a9dc76fdbaf8bd973430146e7742c626531d8d81b7c64d326ce806d2"}
Jul 08 23:10:48.773495 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:48.773166    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv"
Jul 08 23:10:48.791194 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:48.791155    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv" podStartSLOduration=1.249755396 podStartE2EDuration="3.791139262s" totalImagesPullingTime="2.541383866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:10:48.78972148 +0000 UTC m=+467.719755456" watchObservedRunningTime="2026-07-08 23:10:48.791139262 +0000 UTC m=+467.721173238"
Jul 08 23:10:49.993856 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:49.993822    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:50.448364 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.448336    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"]
Jul 08 23:10:50.451681 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.451666    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:10:50.454121 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.454103    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-b6jvt\""
Jul 08 23:10:50.463597 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.463566    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"]
Jul 08 23:10:50.545166 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.545137    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnpw4\" (UniqueName: \"kubernetes.io/projected/dcafbaa5-aa4f-4098-84f1-915489a59fa3-kube-api-access-cnpw4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" (UID: \"dcafbaa5-aa4f-4098-84f1-915489a59fa3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:10:50.545336 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.545190    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcafbaa5-aa4f-4098-84f1-915489a59fa3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" (UID: \"dcafbaa5-aa4f-4098-84f1-915489a59fa3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:10:50.646199 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.646156    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcafbaa5-aa4f-4098-84f1-915489a59fa3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" (UID: \"dcafbaa5-aa4f-4098-84f1-915489a59fa3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:10:50.646344 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.646257    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnpw4\" (UniqueName: \"kubernetes.io/projected/dcafbaa5-aa4f-4098-84f1-915489a59fa3-kube-api-access-cnpw4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" (UID: \"dcafbaa5-aa4f-4098-84f1-915489a59fa3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:10:50.646631 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.646612    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcafbaa5-aa4f-4098-84f1-915489a59fa3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" (UID: \"dcafbaa5-aa4f-4098-84f1-915489a59fa3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:10:50.654173 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.654151    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnpw4\" (UniqueName: \"kubernetes.io/projected/dcafbaa5-aa4f-4098-84f1-915489a59fa3-kube-api-access-cnpw4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" (UID: \"dcafbaa5-aa4f-4098-84f1-915489a59fa3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:10:50.761513 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.761403    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:10:50.892043 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:50.892013    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"]
Jul 08 23:10:50.895006 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:10:50.894978    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcafbaa5_aa4f_4098_84f1_915489a59fa3.slice/crio-22f244003a42465eebe3fb9976bb8fed1919795b10c6721c79ce1d051d9c9791 WatchSource:0}: Error finding container 22f244003a42465eebe3fb9976bb8fed1919795b10c6721c79ce1d051d9c9791: Status 404 returned error can't find the container with id 22f244003a42465eebe3fb9976bb8fed1919795b10c6721c79ce1d051d9c9791
Jul 08 23:10:51.147837 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:51.147756    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:51.263272 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:51.263245    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:51.398191 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:51.398120    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:51.787833 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:51.787797    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc" event={"ID":"dcafbaa5-aa4f-4098-84f1-915489a59fa3","Type":"ContainerStarted","Data":"22f244003a42465eebe3fb9976bb8fed1919795b10c6721c79ce1d051d9c9791"}
Jul 08 23:10:51.956387 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:51.956356    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:52.060245 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:52.060170    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:52.357620 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:52.357527    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:52.457630 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:52.457583    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:55.778785 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:55.778750    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:55.882514 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:55.882459    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:57.814631 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:57.814593    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc" event={"ID":"dcafbaa5-aa4f-4098-84f1-915489a59fa3","Type":"ContainerStarted","Data":"f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f"}
Jul 08 23:10:57.815003 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:57.814730    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:10:57.835964 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:57.835871    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc" podStartSLOduration=1.13535471 podStartE2EDuration="7.835850939s" totalImagesPullingTime="6.700496229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:10:57.834402741 +0000 UTC m=+476.764436714" watchObservedRunningTime="2026-07-08 23:10:57.835850939 +0000 UTC m=+476.765884914"
Jul 08 23:10:59.162512 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:59.162409    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:59.220708 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:59.220680    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:10:59.779425 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:10:59.779400    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-249lv"
Jul 08 23:11:00.434183 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.434145    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"]
Jul 08 23:11:00.437457 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.437438    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"
Jul 08 23:11:00.439873 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.439848    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 08 23:11:00.439973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.439906    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 08 23:11:00.439973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.439931    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lh2pm\""
Jul 08 23:11:00.445784 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.445767    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"]
Jul 08 23:11:00.448380 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.448352    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:00.538164 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.538136    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zffdt\" (UniqueName: \"kubernetes.io/projected/d6a7e345-1bc1-44fa-9bf0-a91d25622140-kube-api-access-zffdt\") pod \"kuadrant-console-plugin-66b5774985-p5z5l\" (UID: \"d6a7e345-1bc1-44fa-9bf0-a91d25622140\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"
Jul 08 23:11:00.538300 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.538175    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d6a7e345-1bc1-44fa-9bf0-a91d25622140-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-p5z5l\" (UID: \"d6a7e345-1bc1-44fa-9bf0-a91d25622140\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"
Jul 08 23:11:00.538300 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.538235    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6a7e345-1bc1-44fa-9bf0-a91d25622140-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-p5z5l\" (UID: \"d6a7e345-1bc1-44fa-9bf0-a91d25622140\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"
Jul 08 23:11:00.639636 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.639595    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d6a7e345-1bc1-44fa-9bf0-a91d25622140-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-p5z5l\" (UID: \"d6a7e345-1bc1-44fa-9bf0-a91d25622140\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"
Jul 08 23:11:00.639828 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.639694    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6a7e345-1bc1-44fa-9bf0-a91d25622140-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-p5z5l\" (UID: \"d6a7e345-1bc1-44fa-9bf0-a91d25622140\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"
Jul 08 23:11:00.639828 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.639751    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zffdt\" (UniqueName: \"kubernetes.io/projected/d6a7e345-1bc1-44fa-9bf0-a91d25622140-kube-api-access-zffdt\") pod \"kuadrant-console-plugin-66b5774985-p5z5l\" (UID: \"d6a7e345-1bc1-44fa-9bf0-a91d25622140\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"
Jul 08 23:11:00.640351 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.640330    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d6a7e345-1bc1-44fa-9bf0-a91d25622140-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-p5z5l\" (UID: \"d6a7e345-1bc1-44fa-9bf0-a91d25622140\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"
Jul 08 23:11:00.642046 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.642024    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6a7e345-1bc1-44fa-9bf0-a91d25622140-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-p5z5l\" (UID: \"d6a7e345-1bc1-44fa-9bf0-a91d25622140\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"
Jul 08 23:11:00.646980 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.646960    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:00.649390 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.649371    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zffdt\" (UniqueName: \"kubernetes.io/projected/d6a7e345-1bc1-44fa-9bf0-a91d25622140-kube-api-access-zffdt\") pod \"kuadrant-console-plugin-66b5774985-p5z5l\" (UID: \"d6a7e345-1bc1-44fa-9bf0-a91d25622140\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"
Jul 08 23:11:00.747284 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.747221    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"
Jul 08 23:11:00.870450 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:00.870409    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l"]
Jul 08 23:11:00.874280 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:11:00.874254    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6a7e345_1bc1_44fa_9bf0_a91d25622140.slice/crio-ca0b94dd125755c1b7f6e52bb0836901dbbb76bdcdb83f4fb49d86bb0bb7477a WatchSource:0}: Error finding container ca0b94dd125755c1b7f6e52bb0836901dbbb76bdcdb83f4fb49d86bb0bb7477a: Status 404 returned error can't find the container with id ca0b94dd125755c1b7f6e52bb0836901dbbb76bdcdb83f4fb49d86bb0bb7477a
Jul 08 23:11:01.832866 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:01.832828    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l" event={"ID":"d6a7e345-1bc1-44fa-9bf0-a91d25622140","Type":"ContainerStarted","Data":"ca0b94dd125755c1b7f6e52bb0836901dbbb76bdcdb83f4fb49d86bb0bb7477a"}
Jul 08 23:11:03.145643 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:03.145604    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:03.199749 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:03.199716    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:06.781605 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:06.781571    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:08.821691 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:08.821661    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:11:09.802954 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:09.802911    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"]
Jul 08 23:11:09.809679 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:09.809656    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:09.819009 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:09.818782    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"]
Jul 08 23:11:09.922730 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:09.922685    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9282e476-86f9-472f-a4a6-a4fcece18da6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4msbw\" (UID: \"9282e476-86f9-472f-a4a6-a4fcece18da6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:09.923156 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:09.922799    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb2sc\" (UniqueName: \"kubernetes.io/projected/9282e476-86f9-472f-a4a6-a4fcece18da6-kube-api-access-vb2sc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4msbw\" (UID: \"9282e476-86f9-472f-a4a6-a4fcece18da6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:10.024334 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.024287    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9282e476-86f9-472f-a4a6-a4fcece18da6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4msbw\" (UID: \"9282e476-86f9-472f-a4a6-a4fcece18da6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:10.024560 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.024384    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb2sc\" (UniqueName: \"kubernetes.io/projected/9282e476-86f9-472f-a4a6-a4fcece18da6-kube-api-access-vb2sc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4msbw\" (UID: \"9282e476-86f9-472f-a4a6-a4fcece18da6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:10.024797 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.024760    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9282e476-86f9-472f-a4a6-a4fcece18da6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4msbw\" (UID: \"9282e476-86f9-472f-a4a6-a4fcece18da6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:10.034545 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.034510    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb2sc\" (UniqueName: \"kubernetes.io/projected/9282e476-86f9-472f-a4a6-a4fcece18da6-kube-api-access-vb2sc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4msbw\" (UID: \"9282e476-86f9-472f-a4a6-a4fcece18da6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:10.123701 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.123607    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:10.347750 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.347702    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:10.448060 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.448022    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:10.564113 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.564084    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"]
Jul 08 23:11:10.572309 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.572284    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"]
Jul 08 23:11:10.580498 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.580339    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"]
Jul 08 23:11:10.580653 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.580580    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" containerName="manager" containerID="cri-o://f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f" gracePeriod=2
Jul 08 23:11:10.584988 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.584892    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"]
Jul 08 23:11:10.591887 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.591866    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"]
Jul 08 23:11:10.592021 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.591959    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:10.601506 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:10.601415    2560 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"dcafbaa5-aa4f-4098-84f1-915489a59fa3\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-btwvc\": pods \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:11:10.604320 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:10.604187    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:11:10.604961 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.604941    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"]
Jul 08 23:11:10.606568 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.606546    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"]
Jul 08 23:11:10.607064 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.607007    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" containerName="manager"
Jul 08 23:11:10.607064 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.607026    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" containerName="manager"
Jul 08 23:11:10.607325 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.607110    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" containerName="manager"
Jul 08 23:11:10.611411 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.611391    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:10.622666 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.622639    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"]
Jul 08 23:11:10.645011 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:10.644970    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:11:10.729386 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.729299    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6s6d\" (UID: \"f6b4a38a-06a7-4b13-b325-295cb1d5dbad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:10.729386 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.729347    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wttk6\" (UniqueName: \"kubernetes.io/projected/3483e638-f2cb-4678-8131-f0e9fdfe74b4-kube-api-access-wttk6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8v4j\" (UID: \"3483e638-f2cb-4678-8131-f0e9fdfe74b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:10.729627 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.729439    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4jtj\" (UniqueName: \"kubernetes.io/projected/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-kube-api-access-s4jtj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6s6d\" (UID: \"f6b4a38a-06a7-4b13-b325-295cb1d5dbad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:10.729627 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.729513    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3483e638-f2cb-4678-8131-f0e9fdfe74b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8v4j\" (UID: \"3483e638-f2cb-4678-8131-f0e9fdfe74b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:10.830544 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.830509    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6s6d\" (UID: \"f6b4a38a-06a7-4b13-b325-295cb1d5dbad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:10.830741 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.830561    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wttk6\" (UniqueName: \"kubernetes.io/projected/3483e638-f2cb-4678-8131-f0e9fdfe74b4-kube-api-access-wttk6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8v4j\" (UID: \"3483e638-f2cb-4678-8131-f0e9fdfe74b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:10.830741 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.830603    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4jtj\" (UniqueName: \"kubernetes.io/projected/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-kube-api-access-s4jtj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6s6d\" (UID: \"f6b4a38a-06a7-4b13-b325-295cb1d5dbad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:10.830741 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.830628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3483e638-f2cb-4678-8131-f0e9fdfe74b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8v4j\" (UID: \"3483e638-f2cb-4678-8131-f0e9fdfe74b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:10.830962 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.830936    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6s6d\" (UID: \"f6b4a38a-06a7-4b13-b325-295cb1d5dbad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:10.831119 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.831095    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3483e638-f2cb-4678-8131-f0e9fdfe74b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8v4j\" (UID: \"3483e638-f2cb-4678-8131-f0e9fdfe74b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:10.841301 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.841275    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4jtj\" (UniqueName: \"kubernetes.io/projected/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-kube-api-access-s4jtj\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6s6d\" (UID: \"f6b4a38a-06a7-4b13-b325-295cb1d5dbad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:10.841427 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.841273    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wttk6\" (UniqueName: \"kubernetes.io/projected/3483e638-f2cb-4678-8131-f0e9fdfe74b4-kube-api-access-wttk6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s8v4j\" (UID: \"3483e638-f2cb-4678-8131-f0e9fdfe74b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:10.957716 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.957678    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:10.965563 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.965532    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:10.973262 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:10.973241    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:11.058865 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:11.058738    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:11.552706 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:11.552667    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:11.653138 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:11.653111    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:11.703563 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:11.703528    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:11:14.936208 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:14.936179    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:15.037169 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:15.037126    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:17.340138 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:17.340093    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:17.393960 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:17.393924    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:20.483812 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:20.483764    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:21.890216 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:21.890170    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:21.990674 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:21.990630    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:23.665434 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.665411    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:11:23.668325 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:23.668292    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:11:23.683118 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.683087    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"]
Jul 08 23:11:23.687378 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:11:23.687352    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3483e638_f2cb_4678_8131_f0e9fdfe74b4.slice/crio-3aabfa82366249b8cae306e5db04c0a6d362c63b1ac46eb352e4c1d39c78e396 WatchSource:0}: Error finding container 3aabfa82366249b8cae306e5db04c0a6d362c63b1ac46eb352e4c1d39c78e396: Status 404 returned error can't find the container with id 3aabfa82366249b8cae306e5db04c0a6d362c63b1ac46eb352e4c1d39c78e396
Jul 08 23:11:23.710067 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.710042    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"]
Jul 08 23:11:23.718532 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:11:23.718507    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6b4a38a_06a7_4b13_b325_295cb1d5dbad.slice/crio-0d4dabe02ea33e52e78920b01ac4eede509fddbd2f18983e4ff617623f214581 WatchSource:0}: Error finding container 0d4dabe02ea33e52e78920b01ac4eede509fddbd2f18983e4ff617623f214581: Status 404 returned error can't find the container with id 0d4dabe02ea33e52e78920b01ac4eede509fddbd2f18983e4ff617623f214581
Jul 08 23:11:23.858660 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.858567    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnpw4\" (UniqueName: \"kubernetes.io/projected/dcafbaa5-aa4f-4098-84f1-915489a59fa3-kube-api-access-cnpw4\") pod \"dcafbaa5-aa4f-4098-84f1-915489a59fa3\" (UID: \"dcafbaa5-aa4f-4098-84f1-915489a59fa3\") "
Jul 08 23:11:23.858660 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.858646    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcafbaa5-aa4f-4098-84f1-915489a59fa3-extensions-socket-volume\") pod \"dcafbaa5-aa4f-4098-84f1-915489a59fa3\" (UID: \"dcafbaa5-aa4f-4098-84f1-915489a59fa3\") "
Jul 08 23:11:23.859018 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.858993    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcafbaa5-aa4f-4098-84f1-915489a59fa3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dcafbaa5-aa4f-4098-84f1-915489a59fa3" (UID: "dcafbaa5-aa4f-4098-84f1-915489a59fa3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:11:23.860904 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.860881    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcafbaa5-aa4f-4098-84f1-915489a59fa3-kube-api-access-cnpw4" (OuterVolumeSpecName: "kube-api-access-cnpw4") pod "dcafbaa5-aa4f-4098-84f1-915489a59fa3" (UID: "dcafbaa5-aa4f-4098-84f1-915489a59fa3"). InnerVolumeSpecName "kube-api-access-cnpw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:11:23.952579 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.952545    2560 generic.go:356] "Generic (PLEG): container finished" podID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" containerID="f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f" exitCode=0
Jul 08 23:11:23.952739 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.952599    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:11:23.952739 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.952642    2560 scope.go:117] "RemoveContainer" containerID="f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f"
Jul 08 23:11:23.954444 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.954399    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l" event={"ID":"d6a7e345-1bc1-44fa-9bf0-a91d25622140","Type":"ContainerStarted","Data":"cd4f2aba45f3e39ae7f86bca3361896fa9d5000922d8336f9d5def05bbe2244e"}
Jul 08 23:11:23.954928 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:23.954899    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:11:23.956181 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.956157    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j" event={"ID":"3483e638-f2cb-4678-8131-f0e9fdfe74b4","Type":"ContainerStarted","Data":"9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804"}
Jul 08 23:11:23.956299 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.956208    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j" event={"ID":"3483e638-f2cb-4678-8131-f0e9fdfe74b4","Type":"ContainerStarted","Data":"3aabfa82366249b8cae306e5db04c0a6d362c63b1ac46eb352e4c1d39c78e396"}
Jul 08 23:11:23.956299 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.956236    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:23.958223 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.958203    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d" event={"ID":"f6b4a38a-06a7-4b13-b325-295cb1d5dbad","Type":"ContainerStarted","Data":"3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1"}
Jul 08 23:11:23.958316 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.958238    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d" event={"ID":"f6b4a38a-06a7-4b13-b325-295cb1d5dbad","Type":"ContainerStarted","Data":"0d4dabe02ea33e52e78920b01ac4eede509fddbd2f18983e4ff617623f214581"}
Jul 08 23:11:23.958437 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.958355    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:23.959164 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.959132    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcafbaa5-aa4f-4098-84f1-915489a59fa3-extensions-socket-volume\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:11:23.959164 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.959150    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnpw4\" (UniqueName: \"kubernetes.io/projected/dcafbaa5-aa4f-4098-84f1-915489a59fa3-kube-api-access-cnpw4\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:11:23.961565 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.961550    2560 scope.go:117] "RemoveContainer" containerID="f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f"
Jul 08 23:11:23.961819 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:23.961800    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f\": container with ID starting with f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f not found: ID does not exist" containerID="f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f"
Jul 08 23:11:23.961873 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.961824    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f"} err="failed to get container status \"f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f\": rpc error: code = NotFound desc = could not find container \"f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f\": container with ID starting with f31582909edb347043ede1adb84e9e2d7fef8b2b90275183d3b56040f4a81b0f not found: ID does not exist"
Jul 08 23:11:23.971409 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.971369    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p5z5l" podStartSLOduration=1.229218467 podStartE2EDuration="23.97135813s" totalImagesPullingTime="22.742139663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:11:23.970198893 +0000 UTC m=+502.900232867" watchObservedRunningTime="2026-07-08 23:11:23.97135813 +0000 UTC m=+502.901392104"
Jul 08 23:11:23.972188 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:23.972165    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:11:23.994657 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:23.994609    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j" podStartSLOduration=13.994592266 podStartE2EDuration="13.994592266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:11:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:11:23.994005279 +0000 UTC m=+502.924039254" watchObservedRunningTime="2026-07-08 23:11:23.994592266 +0000 UTC m=+502.924626245"
Jul 08 23:11:23.995861 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:23.995838    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-btwvc\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-btwvc"
Jul 08 23:11:24.012935 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:24.012887    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d" podStartSLOduration=14.012870287 podStartE2EDuration="14.012870287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:11:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:11:24.012102839 +0000 UTC m=+502.942136814" watchObservedRunningTime="2026-07-08 23:11:24.012870287 +0000 UTC m=+502.942904261"
Jul 08 23:11:25.704990 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:25.704951    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcafbaa5-aa4f-4098-84f1-915489a59fa3" path="/var/lib/kubelet/pods/dcafbaa5-aa4f-4098-84f1-915489a59fa3/volumes"
Jul 08 23:11:26.059537 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:26.059499    2560 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-4msbw_kuadrant-system_9282e476-86f9-472f-a4a6-a4fcece18da6_0(231208cd54e46c70a4598d96f06916710795217e593ddfba123b71875ca03fa8): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-4msbw to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 08 23:11:26.059607 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:26.059560    2560 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-4msbw_kuadrant-system_9282e476-86f9-472f-a4a6-a4fcece18da6_0(231208cd54e46c70a4598d96f06916710795217e593ddfba123b71875ca03fa8): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-4msbw to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:26.973639 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:26.973607    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:26.976546 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:26.976514    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4msbw\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="9282e476-86f9-472f-a4a6-a4fcece18da6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:26.979962 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:26.979943    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:26.982097 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:26.982076    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4msbw\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="9282e476-86f9-472f-a4a6-a4fcece18da6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:27.082544 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:27.082509    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vb2sc\" (UniqueName: \"kubernetes.io/projected/9282e476-86f9-472f-a4a6-a4fcece18da6-kube-api-access-vb2sc\") pod \"9282e476-86f9-472f-a4a6-a4fcece18da6\" (UID: \"9282e476-86f9-472f-a4a6-a4fcece18da6\") "
Jul 08 23:11:27.082700 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:27.082557    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9282e476-86f9-472f-a4a6-a4fcece18da6-extensions-socket-volume\") pod \"9282e476-86f9-472f-a4a6-a4fcece18da6\" (UID: \"9282e476-86f9-472f-a4a6-a4fcece18da6\") "
Jul 08 23:11:27.082886 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:27.082862    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9282e476-86f9-472f-a4a6-a4fcece18da6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9282e476-86f9-472f-a4a6-a4fcece18da6" (UID: "9282e476-86f9-472f-a4a6-a4fcece18da6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:11:27.084689 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:27.084668    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9282e476-86f9-472f-a4a6-a4fcece18da6-kube-api-access-vb2sc" (OuterVolumeSpecName: "kube-api-access-vb2sc") pod "9282e476-86f9-472f-a4a6-a4fcece18da6" (UID: "9282e476-86f9-472f-a4a6-a4fcece18da6"). InnerVolumeSpecName "kube-api-access-vb2sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:11:27.184171 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:27.184136    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb2sc\" (UniqueName: \"kubernetes.io/projected/9282e476-86f9-472f-a4a6-a4fcece18da6-kube-api-access-vb2sc\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:11:27.184171 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:27.184166    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9282e476-86f9-472f-a4a6-a4fcece18da6-extensions-socket-volume\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:11:27.704482 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:27.704441    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9282e476-86f9-472f-a4a6-a4fcece18da6" path="/var/lib/kubelet/pods/9282e476-86f9-472f-a4a6-a4fcece18da6/volumes"
Jul 08 23:11:27.978103 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:27.978029    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:27.980113 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:27.980082    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4msbw\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="9282e476-86f9-472f-a4a6-a4fcece18da6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:27.981936 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:27.981909    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4msbw\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="9282e476-86f9-472f-a4a6-a4fcece18da6" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4msbw"
Jul 08 23:11:34.966110 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:34.966071    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:34.966673 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:34.966657    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:35.038176 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:35.038147    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"]
Jul 08 23:11:35.038388 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:35.038366    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j" podUID="3483e638-f2cb-4678-8131-f0e9fdfe74b4" containerName="manager" containerID="cri-o://9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804" gracePeriod=10
Jul 08 23:11:35.287658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:35.287632    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:35.354212 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:35.354187    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3483e638-f2cb-4678-8131-f0e9fdfe74b4-extensions-socket-volume\") pod \"3483e638-f2cb-4678-8131-f0e9fdfe74b4\" (UID: \"3483e638-f2cb-4678-8131-f0e9fdfe74b4\") "
Jul 08 23:11:35.354368 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:35.354230    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wttk6\" (UniqueName: \"kubernetes.io/projected/3483e638-f2cb-4678-8131-f0e9fdfe74b4-kube-api-access-wttk6\") pod \"3483e638-f2cb-4678-8131-f0e9fdfe74b4\" (UID: \"3483e638-f2cb-4678-8131-f0e9fdfe74b4\") "
Jul 08 23:11:35.354605 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:35.354583    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3483e638-f2cb-4678-8131-f0e9fdfe74b4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3483e638-f2cb-4678-8131-f0e9fdfe74b4" (UID: "3483e638-f2cb-4678-8131-f0e9fdfe74b4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:11:35.356153 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:35.356134    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3483e638-f2cb-4678-8131-f0e9fdfe74b4-kube-api-access-wttk6" (OuterVolumeSpecName: "kube-api-access-wttk6") pod "3483e638-f2cb-4678-8131-f0e9fdfe74b4" (UID: "3483e638-f2cb-4678-8131-f0e9fdfe74b4"). InnerVolumeSpecName "kube-api-access-wttk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:11:35.455706 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:35.455664    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3483e638-f2cb-4678-8131-f0e9fdfe74b4-extensions-socket-volume\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:11:35.455706 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:35.455702    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wttk6\" (UniqueName: \"kubernetes.io/projected/3483e638-f2cb-4678-8131-f0e9fdfe74b4-kube-api-access-wttk6\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:11:36.015224 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:36.015147    2560 generic.go:356] "Generic (PLEG): container finished" podID="3483e638-f2cb-4678-8131-f0e9fdfe74b4" containerID="9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804" exitCode=0
Jul 08 23:11:36.015224 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:36.015186    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"
Jul 08 23:11:36.015224 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:36.015195    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j" event={"ID":"3483e638-f2cb-4678-8131-f0e9fdfe74b4","Type":"ContainerDied","Data":"9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804"}
Jul 08 23:11:36.015726 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:36.015237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j" event={"ID":"3483e638-f2cb-4678-8131-f0e9fdfe74b4","Type":"ContainerDied","Data":"3aabfa82366249b8cae306e5db04c0a6d362c63b1ac46eb352e4c1d39c78e396"}
Jul 08 23:11:36.015726 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:36.015259    2560 scope.go:117] "RemoveContainer" containerID="9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804"
Jul 08 23:11:36.023855 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:36.023840    2560 scope.go:117] "RemoveContainer" containerID="9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804"
Jul 08 23:11:36.024097 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:36.024079    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804\": container with ID starting with 9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804 not found: ID does not exist" containerID="9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804"
Jul 08 23:11:36.024172 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:36.024108    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804"} err="failed to get container status \"9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804\": rpc error: code = NotFound desc = could not find container \"9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804\": container with ID starting with 9508d3e47149bfcd8c93652f99c9c528dcf5925cfd12e8a1a2bd75cb1ccde804 not found: ID does not exist"
Jul 08 23:11:36.033010 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:36.032989    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"]
Jul 08 23:11:36.039914 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:36.039895    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s8v4j"]
Jul 08 23:11:37.704150 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:37.704119    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3483e638-f2cb-4678-8131-f0e9fdfe74b4" path="/var/lib/kubelet/pods/3483e638-f2cb-4678-8131-f0e9fdfe74b4/volumes"
Jul 08 23:11:51.313712 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:51.313680    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:51.413290 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:51.413258    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:11:52.307456 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:52.307427    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"]
Jul 08 23:11:52.307687 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:52.307665    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d" podUID="f6b4a38a-06a7-4b13-b325-295cb1d5dbad" containerName="manager" containerID="cri-o://3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1" gracePeriod=10
Jul 08 23:11:52.553991 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:52.553966    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:52.707585 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:52.707557    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-extensions-socket-volume\") pod \"f6b4a38a-06a7-4b13-b325-295cb1d5dbad\" (UID: \"f6b4a38a-06a7-4b13-b325-295cb1d5dbad\") "
Jul 08 23:11:52.707777 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:52.707619    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4jtj\" (UniqueName: \"kubernetes.io/projected/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-kube-api-access-s4jtj\") pod \"f6b4a38a-06a7-4b13-b325-295cb1d5dbad\" (UID: \"f6b4a38a-06a7-4b13-b325-295cb1d5dbad\") "
Jul 08 23:11:52.707997 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:52.707972    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f6b4a38a-06a7-4b13-b325-295cb1d5dbad" (UID: "f6b4a38a-06a7-4b13-b325-295cb1d5dbad"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:11:52.709661 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:52.709635    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-kube-api-access-s4jtj" (OuterVolumeSpecName: "kube-api-access-s4jtj") pod "f6b4a38a-06a7-4b13-b325-295cb1d5dbad" (UID: "f6b4a38a-06a7-4b13-b325-295cb1d5dbad"). InnerVolumeSpecName "kube-api-access-s4jtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:11:52.808630 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:52.808594    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4jtj\" (UniqueName: \"kubernetes.io/projected/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-kube-api-access-s4jtj\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:11:52.808630 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:52.808624    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6b4a38a-06a7-4b13-b325-295cb1d5dbad-extensions-socket-volume\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:11:53.090461 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:53.090351    2560 generic.go:356] "Generic (PLEG): container finished" podID="f6b4a38a-06a7-4b13-b325-295cb1d5dbad" containerID="3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1" exitCode=0
Jul 08 23:11:53.090461 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:53.090398    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"
Jul 08 23:11:53.090461 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:53.090437    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d" event={"ID":"f6b4a38a-06a7-4b13-b325-295cb1d5dbad","Type":"ContainerDied","Data":"3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1"}
Jul 08 23:11:53.090708 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:53.090507    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d" event={"ID":"f6b4a38a-06a7-4b13-b325-295cb1d5dbad","Type":"ContainerDied","Data":"0d4dabe02ea33e52e78920b01ac4eede509fddbd2f18983e4ff617623f214581"}
Jul 08 23:11:53.090708 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:53.090537    2560 scope.go:117] "RemoveContainer" containerID="3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1"
Jul 08 23:11:53.098888 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:53.098872    2560 scope.go:117] "RemoveContainer" containerID="3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1"
Jul 08 23:11:53.099126 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:11:53.099110    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1\": container with ID starting with 3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1 not found: ID does not exist" containerID="3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1"
Jul 08 23:11:53.099170 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:53.099137    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1"} err="failed to get container status \"3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1\": rpc error: code = NotFound desc = could not find container \"3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1\": container with ID starting with 3fce78e977ba2e9276ff859236a8e3042448e1fffafbc1df30fd957a908417f1 not found: ID does not exist"
Jul 08 23:11:53.111996 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:53.111904    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"]
Jul 08 23:11:53.118215 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:53.118192    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6s6d"]
Jul 08 23:11:53.704141 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:11:53.704109    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6b4a38a-06a7-4b13-b325-295cb1d5dbad" path="/var/lib/kubelet/pods/f6b4a38a-06a7-4b13-b325-295cb1d5dbad/volumes"
Jul 08 23:12:11.143667 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.143625    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:11.155858 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.155831    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f295f"]
Jul 08 23:12:11.156153 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.156142    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6b4a38a-06a7-4b13-b325-295cb1d5dbad" containerName="manager"
Jul 08 23:12:11.156194 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.156155    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6b4a38a-06a7-4b13-b325-295cb1d5dbad" containerName="manager"
Jul 08 23:12:11.156194 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.156172    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3483e638-f2cb-4678-8131-f0e9fdfe74b4" containerName="manager"
Jul 08 23:12:11.156194 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.156178    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3483e638-f2cb-4678-8131-f0e9fdfe74b4" containerName="manager"
Jul 08 23:12:11.156282 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.156236    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6b4a38a-06a7-4b13-b325-295cb1d5dbad" containerName="manager"
Jul 08 23:12:11.156282 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.156247    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3483e638-f2cb-4678-8131-f0e9fdfe74b4" containerName="manager"
Jul 08 23:12:11.190308 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.190277    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f295f"]
Jul 08 23:12:11.190482 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.190385    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:11.193246 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.193227    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 08 23:12:11.247539 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.247506    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f295f"]
Jul 08 23:12:11.266146 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.266116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:11.279174 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.279151    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-config-file\") pod \"limitador-limitador-64c94fd64f-f295f\" (UID: \"3a40bbbe-74f8-4667-94dd-d7b9f358d08b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:11.279266 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.279179    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d689p\" (UniqueName: \"kubernetes.io/projected/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-kube-api-access-d689p\") pod \"limitador-limitador-64c94fd64f-f295f\" (UID: \"3a40bbbe-74f8-4667-94dd-d7b9f358d08b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:11.380422 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.380384    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-config-file\") pod \"limitador-limitador-64c94fd64f-f295f\" (UID: \"3a40bbbe-74f8-4667-94dd-d7b9f358d08b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:11.380422 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.380419    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d689p\" (UniqueName: \"kubernetes.io/projected/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-kube-api-access-d689p\") pod \"limitador-limitador-64c94fd64f-f295f\" (UID: \"3a40bbbe-74f8-4667-94dd-d7b9f358d08b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:11.380993 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.380976    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-config-file\") pod \"limitador-limitador-64c94fd64f-f295f\" (UID: \"3a40bbbe-74f8-4667-94dd-d7b9f358d08b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:11.383394 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.383369    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:11.392290 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.392266    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d689p\" (UniqueName: \"kubernetes.io/projected/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-kube-api-access-d689p\") pod \"limitador-limitador-64c94fd64f-f295f\" (UID: \"3a40bbbe-74f8-4667-94dd-d7b9f358d08b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:11.499514 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.499488    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:11.623741 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.623719    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f295f"]
Jul 08 23:12:11.626166 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:12:11.626139    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a40bbbe_74f8_4667_94dd_d7b9f358d08b.slice/crio-98aa66995d8f333392c4ce0632c5ca824f80ef33414b244cdfe5e986863cf4ac WatchSource:0}: Error finding container 98aa66995d8f333392c4ce0632c5ca824f80ef33414b244cdfe5e986863cf4ac: Status 404 returned error can't find the container with id 98aa66995d8f333392c4ce0632c5ca824f80ef33414b244cdfe5e986863cf4ac
Jul 08 23:12:11.977617 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.977580    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-xbpv5"]
Jul 08 23:12:11.982079 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.982060    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xbpv5"
Jul 08 23:12:11.984414 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.984398    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4thbd\""
Jul 08 23:12:11.989390 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:11.989360    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xbpv5"]
Jul 08 23:12:12.010304 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.010279    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:12.086610 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.086582    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8s4d\" (UniqueName: \"kubernetes.io/projected/e872f02e-a71b-425f-9ec9-14b1d98bed60-kube-api-access-k8s4d\") pod \"authorino-58cd955675-xbpv5\" (UID: \"e872f02e-a71b-425f-9ec9-14b1d98bed60\") " pod="kuadrant-system/authorino-58cd955675-xbpv5"
Jul 08 23:12:12.110810 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.110780    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:12.155635 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.155603    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcctg"]
Jul 08 23:12:12.158757 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.158742    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kcctg"
Jul 08 23:12:12.163163 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.163141    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcctg"]
Jul 08 23:12:12.172381 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.172302    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f" event={"ID":"3a40bbbe-74f8-4667-94dd-d7b9f358d08b","Type":"ContainerStarted","Data":"98aa66995d8f333392c4ce0632c5ca824f80ef33414b244cdfe5e986863cf4ac"}
Jul 08 23:12:12.187307 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.187284    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8s4d\" (UniqueName: \"kubernetes.io/projected/e872f02e-a71b-425f-9ec9-14b1d98bed60-kube-api-access-k8s4d\") pod \"authorino-58cd955675-xbpv5\" (UID: \"e872f02e-a71b-425f-9ec9-14b1d98bed60\") " pod="kuadrant-system/authorino-58cd955675-xbpv5"
Jul 08 23:12:12.187396 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.187321    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h59hn\" (UniqueName: \"kubernetes.io/projected/acb1e545-e5d9-4299-b90d-91623528d44c-kube-api-access-h59hn\") pod \"authorino-75d556bd77-kcctg\" (UID: \"acb1e545-e5d9-4299-b90d-91623528d44c\") " pod="kuadrant-system/authorino-75d556bd77-kcctg"
Jul 08 23:12:12.194971 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.194937    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8s4d\" (UniqueName: \"kubernetes.io/projected/e872f02e-a71b-425f-9ec9-14b1d98bed60-kube-api-access-k8s4d\") pod \"authorino-58cd955675-xbpv5\" (UID: \"e872f02e-a71b-425f-9ec9-14b1d98bed60\") " pod="kuadrant-system/authorino-58cd955675-xbpv5"
Jul 08 23:12:12.288519 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.288445    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h59hn\" (UniqueName: \"kubernetes.io/projected/acb1e545-e5d9-4299-b90d-91623528d44c-kube-api-access-h59hn\") pod \"authorino-75d556bd77-kcctg\" (UID: \"acb1e545-e5d9-4299-b90d-91623528d44c\") " pod="kuadrant-system/authorino-75d556bd77-kcctg"
Jul 08 23:12:12.292375 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.292350    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xbpv5"
Jul 08 23:12:12.297326 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.297305    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h59hn\" (UniqueName: \"kubernetes.io/projected/acb1e545-e5d9-4299-b90d-91623528d44c-kube-api-access-h59hn\") pod \"authorino-75d556bd77-kcctg\" (UID: \"acb1e545-e5d9-4299-b90d-91623528d44c\") " pod="kuadrant-system/authorino-75d556bd77-kcctg"
Jul 08 23:12:12.416515 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.416489    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xbpv5"]
Jul 08 23:12:12.418918 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:12:12.418892    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode872f02e_a71b_425f_9ec9_14b1d98bed60.slice/crio-fe77a339cef14d1b3e9b1af4dfa9f4b27749128c69545edb986eabb5a32f97be WatchSource:0}: Error finding container fe77a339cef14d1b3e9b1af4dfa9f4b27749128c69545edb986eabb5a32f97be: Status 404 returned error can't find the container with id fe77a339cef14d1b3e9b1af4dfa9f4b27749128c69545edb986eabb5a32f97be
Jul 08 23:12:12.467997 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.467975    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kcctg"
Jul 08 23:12:12.623122 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:12.623091    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcctg"]
Jul 08 23:12:12.626656 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:12:12.626431    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacb1e545_e5d9_4299_b90d_91623528d44c.slice/crio-6969045f200d6f5157a15b6d8c8493500d2ef866875cead4ab64657e9ac07c3a WatchSource:0}: Error finding container 6969045f200d6f5157a15b6d8c8493500d2ef866875cead4ab64657e9ac07c3a: Status 404 returned error can't find the container with id 6969045f200d6f5157a15b6d8c8493500d2ef866875cead4ab64657e9ac07c3a
Jul 08 23:12:13.184720 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:13.184682    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kcctg" event={"ID":"acb1e545-e5d9-4299-b90d-91623528d44c","Type":"ContainerStarted","Data":"6969045f200d6f5157a15b6d8c8493500d2ef866875cead4ab64657e9ac07c3a"}
Jul 08 23:12:13.186365 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:13.186337    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xbpv5" event={"ID":"e872f02e-a71b-425f-9ec9-14b1d98bed60","Type":"ContainerStarted","Data":"fe77a339cef14d1b3e9b1af4dfa9f4b27749128c69545edb986eabb5a32f97be"}
Jul 08 23:12:17.219239 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:17.219149    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f" event={"ID":"3a40bbbe-74f8-4667-94dd-d7b9f358d08b","Type":"ContainerStarted","Data":"078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9"}
Jul 08 23:12:17.219239 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:17.219236    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:17.220704 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:17.220669    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kcctg" event={"ID":"acb1e545-e5d9-4299-b90d-91623528d44c","Type":"ContainerStarted","Data":"d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695"}
Jul 08 23:12:17.221972 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:17.221941    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xbpv5" event={"ID":"e872f02e-a71b-425f-9ec9-14b1d98bed60","Type":"ContainerStarted","Data":"a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f"}
Jul 08 23:12:17.235300 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:17.235258    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f" podStartSLOduration=0.938634395 podStartE2EDuration="6.235246552s" totalImagesPullingTime="5.296612157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:12:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:12:17.234103162 +0000 UTC m=+556.164137147" watchObservedRunningTime="2026-07-08 23:12:17.235246552 +0000 UTC m=+556.165280526"
Jul 08 23:12:17.247791 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:17.247752    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-xbpv5" podStartSLOduration=1.755365345 podStartE2EDuration="6.247740026s" totalImagesPullingTime="4.492374681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:12:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:12:17.246525867 +0000 UTC m=+556.176559842" watchObservedRunningTime="2026-07-08 23:12:17.247740026 +0000 UTC m=+556.177773999"
Jul 08 23:12:17.261661 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:17.261611    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-kcctg" podStartSLOduration=1.01893116 podStartE2EDuration="5.261593482s" totalImagesPullingTime="4.242662322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:12:17.259709477 +0000 UTC m=+556.189743452" watchObservedRunningTime="2026-07-08 23:12:17.261593482 +0000 UTC m=+556.191627457"
Jul 08 23:12:17.282695 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:17.282667    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xbpv5"]
Jul 08 23:12:19.230747 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:19.230713    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-xbpv5" podUID="e872f02e-a71b-425f-9ec9-14b1d98bed60" containerName="authorino" containerID="cri-o://a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f" gracePeriod=30
Jul 08 23:12:19.483011 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:19.482955    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xbpv5"
Jul 08 23:12:19.555170 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:19.555136    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8s4d\" (UniqueName: \"kubernetes.io/projected/e872f02e-a71b-425f-9ec9-14b1d98bed60-kube-api-access-k8s4d\") pod \"e872f02e-a71b-425f-9ec9-14b1d98bed60\" (UID: \"e872f02e-a71b-425f-9ec9-14b1d98bed60\") "
Jul 08 23:12:19.557102 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:19.557074    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e872f02e-a71b-425f-9ec9-14b1d98bed60-kube-api-access-k8s4d" (OuterVolumeSpecName: "kube-api-access-k8s4d") pod "e872f02e-a71b-425f-9ec9-14b1d98bed60" (UID: "e872f02e-a71b-425f-9ec9-14b1d98bed60"). InnerVolumeSpecName "kube-api-access-k8s4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:12:19.655993 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:19.655951    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8s4d\" (UniqueName: \"kubernetes.io/projected/e872f02e-a71b-425f-9ec9-14b1d98bed60-kube-api-access-k8s4d\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:12:20.236279 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:20.236247    2560 generic.go:356] "Generic (PLEG): container finished" podID="e872f02e-a71b-425f-9ec9-14b1d98bed60" containerID="a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f" exitCode=0
Jul 08 23:12:20.236681 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:20.236288    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xbpv5"
Jul 08 23:12:20.236681 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:20.236291    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xbpv5" event={"ID":"e872f02e-a71b-425f-9ec9-14b1d98bed60","Type":"ContainerDied","Data":"a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f"}
Jul 08 23:12:20.236681 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:20.236379    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xbpv5" event={"ID":"e872f02e-a71b-425f-9ec9-14b1d98bed60","Type":"ContainerDied","Data":"fe77a339cef14d1b3e9b1af4dfa9f4b27749128c69545edb986eabb5a32f97be"}
Jul 08 23:12:20.236681 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:20.236395    2560 scope.go:117] "RemoveContainer" containerID="a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f"
Jul 08 23:12:20.244248 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:20.244231    2560 scope.go:117] "RemoveContainer" containerID="a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f"
Jul 08 23:12:20.244494 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:12:20.244460    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f\": container with ID starting with a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f not found: ID does not exist" containerID="a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f"
Jul 08 23:12:20.244542 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:20.244503    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f"} err="failed to get container status \"a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f\": rpc error: code = NotFound desc = could not find container \"a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f\": container with ID starting with a256ba004925ce33baf313a7535724b69c7cc8f30ebc04bb0cb6491219a8dd5f not found: ID does not exist"
Jul 08 23:12:20.250752 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:20.250733    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xbpv5"]
Jul 08 23:12:20.252895 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:20.252877    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-xbpv5"]
Jul 08 23:12:21.703484 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:21.703442    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e872f02e-a71b-425f-9ec9-14b1d98bed60" path="/var/lib/kubelet/pods/e872f02e-a71b-425f-9ec9-14b1d98bed60/volumes"
Jul 08 23:12:26.486810 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:26.486776    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f295f"]
Jul 08 23:12:26.487209 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:26.486950    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f" podUID="3a40bbbe-74f8-4667-94dd-d7b9f358d08b" containerName="limitador" containerID="cri-o://078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9" gracePeriod=30
Jul 08 23:12:26.487629 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:26.487611    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:27.031434 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.031412    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:27.122059 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.121983    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d689p\" (UniqueName: \"kubernetes.io/projected/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-kube-api-access-d689p\") pod \"3a40bbbe-74f8-4667-94dd-d7b9f358d08b\" (UID: \"3a40bbbe-74f8-4667-94dd-d7b9f358d08b\") "
Jul 08 23:12:27.122201 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.122070    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-config-file\") pod \"3a40bbbe-74f8-4667-94dd-d7b9f358d08b\" (UID: \"3a40bbbe-74f8-4667-94dd-d7b9f358d08b\") "
Jul 08 23:12:27.122394 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.122373    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-config-file" (OuterVolumeSpecName: "config-file") pod "3a40bbbe-74f8-4667-94dd-d7b9f358d08b" (UID: "3a40bbbe-74f8-4667-94dd-d7b9f358d08b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:12:27.124002 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.123978    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-kube-api-access-d689p" (OuterVolumeSpecName: "kube-api-access-d689p") pod "3a40bbbe-74f8-4667-94dd-d7b9f358d08b" (UID: "3a40bbbe-74f8-4667-94dd-d7b9f358d08b"). InnerVolumeSpecName "kube-api-access-d689p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:12:27.223521 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.223495    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d689p\" (UniqueName: \"kubernetes.io/projected/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-kube-api-access-d689p\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:12:27.223521 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.223518    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3a40bbbe-74f8-4667-94dd-d7b9f358d08b-config-file\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:12:27.268252 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.268227    2560 generic.go:356] "Generic (PLEG): container finished" podID="3a40bbbe-74f8-4667-94dd-d7b9f358d08b" containerID="078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9" exitCode=0
Jul 08 23:12:27.268373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.268271    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f"
Jul 08 23:12:27.268373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.268281    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f" event={"ID":"3a40bbbe-74f8-4667-94dd-d7b9f358d08b","Type":"ContainerDied","Data":"078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9"}
Jul 08 23:12:27.268373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.268323    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f295f" event={"ID":"3a40bbbe-74f8-4667-94dd-d7b9f358d08b","Type":"ContainerDied","Data":"98aa66995d8f333392c4ce0632c5ca824f80ef33414b244cdfe5e986863cf4ac"}
Jul 08 23:12:27.268373 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.268344    2560 scope.go:117] "RemoveContainer" containerID="078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9"
Jul 08 23:12:27.276769 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.276753    2560 scope.go:117] "RemoveContainer" containerID="078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9"
Jul 08 23:12:27.277000 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:12:27.276981    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9\": container with ID starting with 078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9 not found: ID does not exist" containerID="078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9"
Jul 08 23:12:27.277047 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.277006    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9"} err="failed to get container status \"078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9\": rpc error: code = NotFound desc = could not find container \"078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9\": container with ID starting with 078cbc807a24e75d38451de04efbddcadb700de915193da10664f500c25f6df9 not found: ID does not exist"
Jul 08 23:12:27.288024 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.288000    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f295f"]
Jul 08 23:12:27.294079 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.294061    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f295f"]
Jul 08 23:12:27.703858 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:27.703819    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a40bbbe-74f8-4667-94dd-d7b9f358d08b" path="/var/lib/kubelet/pods/3a40bbbe-74f8-4667-94dd-d7b9f358d08b/volumes"
Jul 08 23:12:29.868813 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:29.868779    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:29.968175 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:29.968143    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:30.347330 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.347296    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-psmps"]
Jul 08 23:12:30.347662 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.347650    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e872f02e-a71b-425f-9ec9-14b1d98bed60" containerName="authorino"
Jul 08 23:12:30.347733 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.347664    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e872f02e-a71b-425f-9ec9-14b1d98bed60" containerName="authorino"
Jul 08 23:12:30.347733 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.347690    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a40bbbe-74f8-4667-94dd-d7b9f358d08b" containerName="limitador"
Jul 08 23:12:30.347733 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.347696    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a40bbbe-74f8-4667-94dd-d7b9f358d08b" containerName="limitador"
Jul 08 23:12:30.347842 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.347752    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e872f02e-a71b-425f-9ec9-14b1d98bed60" containerName="authorino"
Jul 08 23:12:30.347842 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.347761    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a40bbbe-74f8-4667-94dd-d7b9f358d08b" containerName="limitador"
Jul 08 23:12:30.350869 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.350855    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps"
Jul 08 23:12:30.354545 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.354524    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-47sht\""
Jul 08 23:12:30.354669 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.354618    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 08 23:12:30.354768 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.354754    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 08 23:12:30.355397 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.355379    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 08 23:12:30.363110 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.363079    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-psmps"]
Jul 08 23:12:30.372204 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.372184    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:30.449584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.449547    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3b2851d0-57c0-42c1-ac08-56d0148894ba-data\") pod \"postgres-69fb4594cf-psmps\" (UID: \"3b2851d0-57c0-42c1-ac08-56d0148894ba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps"
Jul 08 23:12:30.449720 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.449683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvshq\" (UniqueName: \"kubernetes.io/projected/3b2851d0-57c0-42c1-ac08-56d0148894ba-kube-api-access-cvshq\") pod \"postgres-69fb4594cf-psmps\" (UID: \"3b2851d0-57c0-42c1-ac08-56d0148894ba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps"
Jul 08 23:12:30.551061 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.551030    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3b2851d0-57c0-42c1-ac08-56d0148894ba-data\") pod \"postgres-69fb4594cf-psmps\" (UID: \"3b2851d0-57c0-42c1-ac08-56d0148894ba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps"
Jul 08 23:12:30.551217 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.551115    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvshq\" (UniqueName: \"kubernetes.io/projected/3b2851d0-57c0-42c1-ac08-56d0148894ba-kube-api-access-cvshq\") pod \"postgres-69fb4594cf-psmps\" (UID: \"3b2851d0-57c0-42c1-ac08-56d0148894ba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps"
Jul 08 23:12:30.551427 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.551397    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3b2851d0-57c0-42c1-ac08-56d0148894ba-data\") pod \"postgres-69fb4594cf-psmps\" (UID: \"3b2851d0-57c0-42c1-ac08-56d0148894ba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps"
Jul 08 23:12:30.559102 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.559076    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvshq\" (UniqueName: \"kubernetes.io/projected/3b2851d0-57c0-42c1-ac08-56d0148894ba-kube-api-access-cvshq\") pod \"postgres-69fb4594cf-psmps\" (UID: \"3b2851d0-57c0-42c1-ac08-56d0148894ba\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps"
Jul 08 23:12:30.661238 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.661172    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps"
Jul 08 23:12:30.786280 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:30.786256    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-psmps"]
Jul 08 23:12:30.788530 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:12:30.788503    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b2851d0_57c0_42c1_ac08_56d0148894ba.slice/crio-ab463ad5b6ac23397510b1d09be4d7070d474a48ea65a8f0feb5a09cca875680 WatchSource:0}: Error finding container ab463ad5b6ac23397510b1d09be4d7070d474a48ea65a8f0feb5a09cca875680: Status 404 returned error can't find the container with id ab463ad5b6ac23397510b1d09be4d7070d474a48ea65a8f0feb5a09cca875680
Jul 08 23:12:31.288519 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:31.288484    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps" event={"ID":"3b2851d0-57c0-42c1-ac08-56d0148894ba","Type":"ContainerStarted","Data":"ab463ad5b6ac23397510b1d09be4d7070d474a48ea65a8f0feb5a09cca875680"}
Jul 08 23:12:38.323278 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:38.323239    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps" event={"ID":"3b2851d0-57c0-42c1-ac08-56d0148894ba","Type":"ContainerStarted","Data":"a6ea68dcfb702b7bc437451011b5a37a9d21e89db58a9809da0227ff60a28116"}
Jul 08 23:12:38.323728 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:38.323373    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps"
Jul 08 23:12:38.339549 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:38.339480    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps" podStartSLOduration=1.8454760879999998 podStartE2EDuration="8.339449199s" totalImagesPullingTime="6.493973111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:12:38.337803866 +0000 UTC m=+577.267837841" watchObservedRunningTime="2026-07-08 23:12:38.339449199 +0000 UTC m=+577.269483168"
Jul 08 23:12:44.356849 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:44.356815    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-psmps"
Jul 08 23:12:44.644349 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:44.644268    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:44.867732 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:44.867700    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wb2vx"]
Jul 08 23:12:44.874315 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:44.874294    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wb2vx"
Jul 08 23:12:44.877249 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:44.877224    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wb2vx"]
Jul 08 23:12:44.981158 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:44.981122    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flt4p\" (UniqueName: \"kubernetes.io/projected/a49a1fb4-67f5-468e-a064-45718e50f571-kube-api-access-flt4p\") pod \"authorino-5d4b67c446-wb2vx\" (UID: \"a49a1fb4-67f5-468e-a064-45718e50f571\") " pod="kuadrant-system/authorino-5d4b67c446-wb2vx"
Jul 08 23:12:45.052886 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.052845    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wb2vx"]
Jul 08 23:12:45.053148 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:12:45.053124    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-flt4p], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-wb2vx" podUID="a49a1fb4-67f5-468e-a064-45718e50f571"
Jul 08 23:12:45.077168 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.077140    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-vx6c2"]
Jul 08 23:12:45.081125 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.081109    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vx6c2"
Jul 08 23:12:45.081647 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.081629    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flt4p\" (UniqueName: \"kubernetes.io/projected/a49a1fb4-67f5-468e-a064-45718e50f571-kube-api-access-flt4p\") pod \"authorino-5d4b67c446-wb2vx\" (UID: \"a49a1fb4-67f5-468e-a064-45718e50f571\") " pod="kuadrant-system/authorino-5d4b67c446-wb2vx"
Jul 08 23:12:45.083403 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.083383    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 08 23:12:45.088754 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.088733    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-vx6c2"]
Jul 08 23:12:45.092188 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.092158    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flt4p\" (UniqueName: \"kubernetes.io/projected/a49a1fb4-67f5-468e-a064-45718e50f571-kube-api-access-flt4p\") pod \"authorino-5d4b67c446-wb2vx\" (UID: \"a49a1fb4-67f5-468e-a064-45718e50f571\") " pod="kuadrant-system/authorino-5d4b67c446-wb2vx"
Jul 08 23:12:45.107351 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.107328    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-vx6c2"]
Jul 08 23:12:45.107586 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:12:45.107566    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-tjff9 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-vx6c2" podUID="5f2eb759-a2c0-43b0-9508-a026ee5b82fd"
Jul 08 23:12:45.133107 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.133083    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-554c4b5df-sfsmg"]
Jul 08 23:12:45.136805 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.136790    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-554c4b5df-sfsmg"
Jul 08 23:12:45.143117 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.143097    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-554c4b5df-sfsmg"]
Jul 08 23:12:45.182762 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.182740    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-tls-cert\") pod \"authorino-76877994-vx6c2\" (UID: \"5f2eb759-a2c0-43b0-9508-a026ee5b82fd\") " pod="kuadrant-system/authorino-76877994-vx6c2"
Jul 08 23:12:45.182902 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.182791    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjff9\" (UniqueName: \"kubernetes.io/projected/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-kube-api-access-tjff9\") pod \"authorino-76877994-vx6c2\" (UID: \"5f2eb759-a2c0-43b0-9508-a026ee5b82fd\") " pod="kuadrant-system/authorino-76877994-vx6c2"
Jul 08 23:12:45.283868 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.283782    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-tls-cert\") pod \"authorino-554c4b5df-sfsmg\" (UID: \"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0\") " pod="kuadrant-system/authorino-554c4b5df-sfsmg"
Jul 08 23:12:45.283868 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.283815    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9stgm\" (UniqueName: \"kubernetes.io/projected/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-kube-api-access-9stgm\") pod \"authorino-554c4b5df-sfsmg\" (UID: \"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0\") " pod="kuadrant-system/authorino-554c4b5df-sfsmg"
Jul 08 23:12:45.283868 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.283856    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjff9\" (UniqueName: \"kubernetes.io/projected/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-kube-api-access-tjff9\") pod \"authorino-76877994-vx6c2\" (UID: \"5f2eb759-a2c0-43b0-9508-a026ee5b82fd\") " pod="kuadrant-system/authorino-76877994-vx6c2"
Jul 08 23:12:45.284126 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.283937    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-tls-cert\") pod \"authorino-76877994-vx6c2\" (UID: \"5f2eb759-a2c0-43b0-9508-a026ee5b82fd\") " pod="kuadrant-system/authorino-76877994-vx6c2"
Jul 08 23:12:45.286312 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.286288    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-tls-cert\") pod \"authorino-76877994-vx6c2\" (UID: \"5f2eb759-a2c0-43b0-9508-a026ee5b82fd\") " pod="kuadrant-system/authorino-76877994-vx6c2"
Jul 08 23:12:45.291715 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.291692    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjff9\" (UniqueName: \"kubernetes.io/projected/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-kube-api-access-tjff9\") pod \"authorino-76877994-vx6c2\" (UID: \"5f2eb759-a2c0-43b0-9508-a026ee5b82fd\") " pod="kuadrant-system/authorino-76877994-vx6c2"
Jul 08 23:12:45.353430 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.353402    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vx6c2"
Jul 08 23:12:45.353578 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.353402    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wb2vx"
Jul 08 23:12:45.361163 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.361142    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vx6c2"
Jul 08 23:12:45.364099 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.364084    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wb2vx"
Jul 08 23:12:45.384549 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.384516    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-tls-cert\") pod \"authorino-554c4b5df-sfsmg\" (UID: \"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0\") " pod="kuadrant-system/authorino-554c4b5df-sfsmg"
Jul 08 23:12:45.384648 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.384551    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9stgm\" (UniqueName: \"kubernetes.io/projected/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-kube-api-access-9stgm\") pod \"authorino-554c4b5df-sfsmg\" (UID: \"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0\") " pod="kuadrant-system/authorino-554c4b5df-sfsmg"
Jul 08 23:12:45.386812 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.386783    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-tls-cert\") pod \"authorino-554c4b5df-sfsmg\" (UID: \"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0\") " pod="kuadrant-system/authorino-554c4b5df-sfsmg"
Jul 08 23:12:45.392545 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.392522    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9stgm\" (UniqueName: \"kubernetes.io/projected/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-kube-api-access-9stgm\") pod \"authorino-554c4b5df-sfsmg\" (UID: \"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0\") " pod="kuadrant-system/authorino-554c4b5df-sfsmg"
Jul 08 23:12:45.447010 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.446985    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-554c4b5df-sfsmg"
Jul 08 23:12:45.485283 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.485254    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flt4p\" (UniqueName: \"kubernetes.io/projected/a49a1fb4-67f5-468e-a064-45718e50f571-kube-api-access-flt4p\") pod \"a49a1fb4-67f5-468e-a064-45718e50f571\" (UID: \"a49a1fb4-67f5-468e-a064-45718e50f571\") "
Jul 08 23:12:45.486976 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.485306    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjff9\" (UniqueName: \"kubernetes.io/projected/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-kube-api-access-tjff9\") pod \"5f2eb759-a2c0-43b0-9508-a026ee5b82fd\" (UID: \"5f2eb759-a2c0-43b0-9508-a026ee5b82fd\") "
Jul 08 23:12:45.486976 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.485417    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-tls-cert\") pod \"5f2eb759-a2c0-43b0-9508-a026ee5b82fd\" (UID: \"5f2eb759-a2c0-43b0-9508-a026ee5b82fd\") "
Jul 08 23:12:45.487899 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.487861    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-kube-api-access-tjff9" (OuterVolumeSpecName: "kube-api-access-tjff9") pod "5f2eb759-a2c0-43b0-9508-a026ee5b82fd" (UID: "5f2eb759-a2c0-43b0-9508-a026ee5b82fd"). InnerVolumeSpecName "kube-api-access-tjff9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:12:45.488232 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.488200    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5f2eb759-a2c0-43b0-9508-a026ee5b82fd" (UID: "5f2eb759-a2c0-43b0-9508-a026ee5b82fd"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:12:45.488613 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.488592    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a49a1fb4-67f5-468e-a064-45718e50f571-kube-api-access-flt4p" (OuterVolumeSpecName: "kube-api-access-flt4p") pod "a49a1fb4-67f5-468e-a064-45718e50f571" (UID: "a49a1fb4-67f5-468e-a064-45718e50f571"). InnerVolumeSpecName "kube-api-access-flt4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:12:45.570777 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.570752    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-554c4b5df-sfsmg"]
Jul 08 23:12:45.573086 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:12:45.573052    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a4f5535_cfe8_4bf4_86d7_a9ff44b156c0.slice/crio-a81406b2fd030794f5d52e53b33afba52b08c96ec83d22c10467a814c7ee60de WatchSource:0}: Error finding container a81406b2fd030794f5d52e53b33afba52b08c96ec83d22c10467a814c7ee60de: Status 404 returned error can't find the container with id a81406b2fd030794f5d52e53b33afba52b08c96ec83d22c10467a814c7ee60de
Jul 08 23:12:45.586878 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.586855    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flt4p\" (UniqueName: \"kubernetes.io/projected/a49a1fb4-67f5-468e-a064-45718e50f571-kube-api-access-flt4p\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:12:45.586878 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.586879    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjff9\" (UniqueName: \"kubernetes.io/projected/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-kube-api-access-tjff9\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:12:45.586994 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.586888    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f2eb759-a2c0-43b0-9508-a026ee5b82fd-tls-cert\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:12:45.863365 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:45.863274    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:46.360076 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.360029    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-554c4b5df-sfsmg" event={"ID":"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0","Type":"ContainerStarted","Data":"f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e"}
Jul 08 23:12:46.360076 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.360081    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-554c4b5df-sfsmg" event={"ID":"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0","Type":"ContainerStarted","Data":"a81406b2fd030794f5d52e53b33afba52b08c96ec83d22c10467a814c7ee60de"}
Jul 08 23:12:46.360354 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.360151    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-wb2vx"
Jul 08 23:12:46.360354 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.360267    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vx6c2"
Jul 08 23:12:46.375150 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.375108    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-554c4b5df-sfsmg" podStartSLOduration=0.951557063 podStartE2EDuration="1.375094755s" totalImagesPullingTime="423.537692ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:12:46.374620254 +0000 UTC m=+585.304654230" watchObservedRunningTime="2026-07-08 23:12:46.375094755 +0000 UTC m=+585.305128729"
Jul 08 23:12:46.400134 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.400104    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wb2vx"]
Jul 08 23:12:46.405909 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.405873    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-wb2vx"]
Jul 08 23:12:46.407266 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.407240    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcctg"]
Jul 08 23:12:46.407448 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.407425    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-kcctg" podUID="acb1e545-e5d9-4299-b90d-91623528d44c" containerName="authorino" containerID="cri-o://d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695" gracePeriod=30
Jul 08 23:12:46.431780 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.431746    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-vx6c2"]
Jul 08 23:12:46.437221 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.437193    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-vx6c2"]
Jul 08 23:12:46.657902 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.657879    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kcctg"
Jul 08 23:12:46.798780 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.798755    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h59hn\" (UniqueName: \"kubernetes.io/projected/acb1e545-e5d9-4299-b90d-91623528d44c-kube-api-access-h59hn\") pod \"acb1e545-e5d9-4299-b90d-91623528d44c\" (UID: \"acb1e545-e5d9-4299-b90d-91623528d44c\") "
Jul 08 23:12:46.800833 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.800807    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acb1e545-e5d9-4299-b90d-91623528d44c-kube-api-access-h59hn" (OuterVolumeSpecName: "kube-api-access-h59hn") pod "acb1e545-e5d9-4299-b90d-91623528d44c" (UID: "acb1e545-e5d9-4299-b90d-91623528d44c"). InnerVolumeSpecName "kube-api-access-h59hn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:12:46.900124 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:46.900044    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h59hn\" (UniqueName: \"kubernetes.io/projected/acb1e545-e5d9-4299-b90d-91623528d44c-kube-api-access-h59hn\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:12:47.366586 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.366552    2560 generic.go:356] "Generic (PLEG): container finished" podID="acb1e545-e5d9-4299-b90d-91623528d44c" containerID="d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695" exitCode=0
Jul 08 23:12:47.366781 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.366594    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kcctg"
Jul 08 23:12:47.366781 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.366635    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kcctg" event={"ID":"acb1e545-e5d9-4299-b90d-91623528d44c","Type":"ContainerDied","Data":"d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695"}
Jul 08 23:12:47.366781 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.366673    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kcctg" event={"ID":"acb1e545-e5d9-4299-b90d-91623528d44c","Type":"ContainerDied","Data":"6969045f200d6f5157a15b6d8c8493500d2ef866875cead4ab64657e9ac07c3a"}
Jul 08 23:12:47.366781 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.366691    2560 scope.go:117] "RemoveContainer" containerID="d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695"
Jul 08 23:12:47.375936 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.375921    2560 scope.go:117] "RemoveContainer" containerID="d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695"
Jul 08 23:12:47.376218 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:12:47.376194    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695\": container with ID starting with d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695 not found: ID does not exist" containerID="d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695"
Jul 08 23:12:47.376257 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.376231    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695"} err="failed to get container status \"d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695\": rpc error: code = NotFound desc = could not find container \"d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695\": container with ID starting with d02fccef80e728d5e1d26f8c0f0f9171ecbd431384b384446fd6d0c471f98695 not found: ID does not exist"
Jul 08 23:12:47.386433 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.386412    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcctg"]
Jul 08 23:12:47.389614 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.389596    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kcctg"]
Jul 08 23:12:47.705858 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.705828    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f2eb759-a2c0-43b0-9508-a026ee5b82fd" path="/var/lib/kubelet/pods/5f2eb759-a2c0-43b0-9508-a026ee5b82fd/volumes"
Jul 08 23:12:47.706083 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.706071    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a49a1fb4-67f5-468e-a064-45718e50f571" path="/var/lib/kubelet/pods/a49a1fb4-67f5-468e-a064-45718e50f571/volumes"
Jul 08 23:12:47.706282 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:47.706271    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acb1e545-e5d9-4299-b90d-91623528d44c" path="/var/lib/kubelet/pods/acb1e545-e5d9-4299-b90d-91623528d44c/volumes"
Jul 08 23:12:50.600113 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.600076    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:50.699484 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.699432    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:50.750401 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.750375    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:50.788175 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.788150    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cdd595cbc-95pnv"]
Jul 08 23:12:50.788516 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.788503    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acb1e545-e5d9-4299-b90d-91623528d44c" containerName="authorino"
Jul 08 23:12:50.788569 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.788517    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="acb1e545-e5d9-4299-b90d-91623528d44c" containerName="authorino"
Jul 08 23:12:50.788604 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.788569    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="acb1e545-e5d9-4299-b90d-91623528d44c" containerName="authorino"
Jul 08 23:12:50.955843 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.955810    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cdd595cbc-95pnv"]
Jul 08 23:12:50.955995 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.955904    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:12:50.959775 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.959741    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n84h8\""
Jul 08 23:12:50.959946 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.959780    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 08 23:12:50.959946 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:50.959823    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 08 23:12:51.135092 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:51.135059    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-webhook-cert\") pod \"maas-controller-6cdd595cbc-95pnv\" (UID: \"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9\") " pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:12:51.135253 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:51.135111    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2v42\" (UniqueName: \"kubernetes.io/projected/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-kube-api-access-q2v42\") pod \"maas-controller-6cdd595cbc-95pnv\" (UID: \"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9\") " pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:12:51.236384 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:51.236315    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-webhook-cert\") pod \"maas-controller-6cdd595cbc-95pnv\" (UID: \"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9\") " pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:12:51.236384 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:51.236362    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2v42\" (UniqueName: \"kubernetes.io/projected/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-kube-api-access-q2v42\") pod \"maas-controller-6cdd595cbc-95pnv\" (UID: \"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9\") " pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:12:51.238695 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:51.238675    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-webhook-cert\") pod \"maas-controller-6cdd595cbc-95pnv\" (UID: \"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9\") " pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:12:51.244497 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:51.244455    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2v42\" (UniqueName: \"kubernetes.io/projected/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-kube-api-access-q2v42\") pod \"maas-controller-6cdd595cbc-95pnv\" (UID: \"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9\") " pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:12:51.265910 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:51.265888    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:12:51.423200 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:51.423172    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cdd595cbc-95pnv"]
Jul 08 23:12:51.425223 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:12:51.425196    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b385482_03ed_4ea3_b0cf_cee7fb69b0b9.slice/crio-86206d2b29a3bd78fc1d677feb7f73182a7ea360d492760d2ddb877b2fff25ad WatchSource:0}: Error finding container 86206d2b29a3bd78fc1d677feb7f73182a7ea360d492760d2ddb877b2fff25ad: Status 404 returned error can't find the container with id 86206d2b29a3bd78fc1d677feb7f73182a7ea360d492760d2ddb877b2fff25ad
Jul 08 23:12:52.394609 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:52.394571    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cdd595cbc-95pnv" event={"ID":"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9","Type":"ContainerStarted","Data":"86206d2b29a3bd78fc1d677feb7f73182a7ea360d492760d2ddb877b2fff25ad"}
Jul 08 23:12:53.401194 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:53.401144    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cdd595cbc-95pnv" event={"ID":"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9","Type":"ContainerStarted","Data":"33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579"}
Jul 08 23:12:53.401695 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:53.401251    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:12:53.425233 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:53.425173    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cdd595cbc-95pnv" podStartSLOduration=1.6026052800000001 podStartE2EDuration="3.425150209s" totalImagesPullingTime="1.822544929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:12:53.422791581 +0000 UTC m=+592.352825568" watchObservedRunningTime="2026-07-08 23:12:53.425150209 +0000 UTC m=+592.355184183"
Jul 08 23:12:53.427860 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:53.427820    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:12:53.511420 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:12:53.511344    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:01.595431 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:01.595403    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:13:01.605022 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:01.604996    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:13:04.410604 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:04.410578    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:13:12.200728 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:12.200682    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:24.921545 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:24.921511    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:25.359503 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.359404    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:25.402890 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.402861    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-785d848678-c4frh"]
Jul 08 23:13:25.406093 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.406077    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:25.412668 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.412650    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 08 23:13:25.413353 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.413332    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tztbd\""
Jul 08 23:13:25.420957 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.420937    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-785d848678-c4frh"]
Jul 08 23:13:25.535876 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.535841    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffhf9\" (UniqueName: \"kubernetes.io/projected/f779c33a-fe24-405b-8285-da64afe92edf-kube-api-access-ffhf9\") pod \"maas-api-785d848678-c4frh\" (UID: \"f779c33a-fe24-405b-8285-da64afe92edf\") " pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:25.535876 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.535880    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f779c33a-fe24-405b-8285-da64afe92edf-maas-api-tls\") pod \"maas-api-785d848678-c4frh\" (UID: \"f779c33a-fe24-405b-8285-da64afe92edf\") " pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:25.536067 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.535971    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f779c33a-fe24-405b-8285-da64afe92edf-openshift-service-ca\") pod \"maas-api-785d848678-c4frh\" (UID: \"f779c33a-fe24-405b-8285-da64afe92edf\") " pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:25.590816 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.590785    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:25.636909 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.636835    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffhf9\" (UniqueName: \"kubernetes.io/projected/f779c33a-fe24-405b-8285-da64afe92edf-kube-api-access-ffhf9\") pod \"maas-api-785d848678-c4frh\" (UID: \"f779c33a-fe24-405b-8285-da64afe92edf\") " pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:25.636909 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.636886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f779c33a-fe24-405b-8285-da64afe92edf-maas-api-tls\") pod \"maas-api-785d848678-c4frh\" (UID: \"f779c33a-fe24-405b-8285-da64afe92edf\") " pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:25.637092 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.636942    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f779c33a-fe24-405b-8285-da64afe92edf-openshift-service-ca\") pod \"maas-api-785d848678-c4frh\" (UID: \"f779c33a-fe24-405b-8285-da64afe92edf\") " pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:25.637507 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.637460    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f779c33a-fe24-405b-8285-da64afe92edf-openshift-service-ca\") pod \"maas-api-785d848678-c4frh\" (UID: \"f779c33a-fe24-405b-8285-da64afe92edf\") " pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:25.639252 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.639233    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f779c33a-fe24-405b-8285-da64afe92edf-maas-api-tls\") pod \"maas-api-785d848678-c4frh\" (UID: \"f779c33a-fe24-405b-8285-da64afe92edf\") " pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:25.645052 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.645028    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffhf9\" (UniqueName: \"kubernetes.io/projected/f779c33a-fe24-405b-8285-da64afe92edf-kube-api-access-ffhf9\") pod \"maas-api-785d848678-c4frh\" (UID: \"f779c33a-fe24-405b-8285-da64afe92edf\") " pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:25.715299 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.715277    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:25.797986 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.797947    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:25.848527 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.848506    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-785d848678-c4frh"]
Jul 08 23:13:25.850620 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:13:25.850594    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf779c33a_fe24_405b_8285_da64afe92edf.slice/crio-7560a176547b0c84a6431309118e75d8c30ccad843d4b8234252246a72cd81d8 WatchSource:0}: Error finding container 7560a176547b0c84a6431309118e75d8c30ccad843d4b8234252246a72cd81d8: Status 404 returned error can't find the container with id 7560a176547b0c84a6431309118e75d8c30ccad843d4b8234252246a72cd81d8
Jul 08 23:13:25.852036 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.852017    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 23:13:25.901747 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.901678    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:25.943802 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.943778    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ksm54"]
Jul 08 23:13:25.948643 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.948622    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-ksm54"
Jul 08 23:13:25.951123 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.951102    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-p5d2q\""
Jul 08 23:13:25.951476 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.951443    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 08 23:13:25.956658 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:25.956637    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ksm54"]
Jul 08 23:13:26.040259 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:26.040224    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a55373e-1a33-4adb-a322-54700ce85504-plugins-config-volume\") pod \"payload-processing-58f86c7798-ksm54\" (UID: \"2a55373e-1a33-4adb-a322-54700ce85504\") " pod="openshift-ingress/payload-processing-58f86c7798-ksm54"
Jul 08 23:13:26.040419 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:26.040300    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxz98\" (UniqueName: \"kubernetes.io/projected/2a55373e-1a33-4adb-a322-54700ce85504-kube-api-access-kxz98\") pod \"payload-processing-58f86c7798-ksm54\" (UID: \"2a55373e-1a33-4adb-a322-54700ce85504\") " pod="openshift-ingress/payload-processing-58f86c7798-ksm54"
Jul 08 23:13:26.141783 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:26.141752    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a55373e-1a33-4adb-a322-54700ce85504-plugins-config-volume\") pod \"payload-processing-58f86c7798-ksm54\" (UID: \"2a55373e-1a33-4adb-a322-54700ce85504\") " pod="openshift-ingress/payload-processing-58f86c7798-ksm54"
Jul 08 23:13:26.141964 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:26.141941    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxz98\" (UniqueName: \"kubernetes.io/projected/2a55373e-1a33-4adb-a322-54700ce85504-kube-api-access-kxz98\") pod \"payload-processing-58f86c7798-ksm54\" (UID: \"2a55373e-1a33-4adb-a322-54700ce85504\") " pod="openshift-ingress/payload-processing-58f86c7798-ksm54"
Jul 08 23:13:26.142581 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:26.142555    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a55373e-1a33-4adb-a322-54700ce85504-plugins-config-volume\") pod \"payload-processing-58f86c7798-ksm54\" (UID: \"2a55373e-1a33-4adb-a322-54700ce85504\") " pod="openshift-ingress/payload-processing-58f86c7798-ksm54"
Jul 08 23:13:26.150146 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:26.150126    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxz98\" (UniqueName: \"kubernetes.io/projected/2a55373e-1a33-4adb-a322-54700ce85504-kube-api-access-kxz98\") pod \"payload-processing-58f86c7798-ksm54\" (UID: \"2a55373e-1a33-4adb-a322-54700ce85504\") " pod="openshift-ingress/payload-processing-58f86c7798-ksm54"
Jul 08 23:13:26.259929 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:26.259898    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-ksm54"
Jul 08 23:13:26.384087 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:26.384054    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ksm54"]
Jul 08 23:13:26.386212 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:13:26.386186    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a55373e_1a33_4adb_a322_54700ce85504.slice/crio-9d89bf452221035949bb8201a2b5c101c81768d26a3a3c85ffaefc42630681e4 WatchSource:0}: Error finding container 9d89bf452221035949bb8201a2b5c101c81768d26a3a3c85ffaefc42630681e4: Status 404 returned error can't find the container with id 9d89bf452221035949bb8201a2b5c101c81768d26a3a3c85ffaefc42630681e4
Jul 08 23:13:26.549521 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:26.549429    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-ksm54" event={"ID":"2a55373e-1a33-4adb-a322-54700ce85504","Type":"ContainerStarted","Data":"9d89bf452221035949bb8201a2b5c101c81768d26a3a3c85ffaefc42630681e4"}
Jul 08 23:13:26.550734 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:26.550708    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh" event={"ID":"f779c33a-fe24-405b-8285-da64afe92edf","Type":"ContainerStarted","Data":"7560a176547b0c84a6431309118e75d8c30ccad843d4b8234252246a72cd81d8"}
Jul 08 23:13:28.561604 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:28.561569    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh" event={"ID":"f779c33a-fe24-405b-8285-da64afe92edf","Type":"ContainerStarted","Data":"3e5834fec07a6f7f7daf93834b72ecf1db084fb5cd73fbd2c9f3c5e37de5c4b6"}
Jul 08 23:13:28.562042 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:28.561633    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:28.563054 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:28.563031    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-ksm54" event={"ID":"2a55373e-1a33-4adb-a322-54700ce85504","Type":"ContainerStarted","Data":"5c2c4f7fec9dddcb6369736c314be17f9f69fdc0564991d44bd897d50474d810"}
Jul 08 23:13:28.563173 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:28.563079    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-ksm54"
Jul 08 23:13:28.577781 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:28.577738    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh" podStartSLOduration=1.871912645 podStartE2EDuration="3.577724403s" totalImagesPullingTime="1.705811758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:13:28.576723605 +0000 UTC m=+627.506757590" watchObservedRunningTime="2026-07-08 23:13:28.577724403 +0000 UTC m=+627.507758424"
Jul 08 23:13:28.593584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:28.593548    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-ksm54" podStartSLOduration=2.064944933 podStartE2EDuration="3.593533963s" totalImagesPullingTime="1.52858903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:13:28.593219293 +0000 UTC m=+627.523253268" watchObservedRunningTime="2026-07-08 23:13:28.593533963 +0000 UTC m=+627.523567937"
Jul 08 23:13:34.573631 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:34.573600    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-785d848678-c4frh"
Jul 08 23:13:36.004047 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:36.004011    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:36.116407 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:36.116374    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:39.570595 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:39.570569    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-ksm54"
Jul 08 23:13:41.628223 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:41.628185    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:42.398936 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.398907    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"]
Jul 08 23:13:42.403611 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.403590    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.405896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.405873    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 08 23:13:42.406001 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.405934    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 08 23:13:42.406946 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.406927    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 08 23:13:42.407061 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.407044    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6p7fv\""
Jul 08 23:13:42.410498 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.410477    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"]
Jul 08 23:13:42.432929 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.432872    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:42.485872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.485844    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.486014 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.485882    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.486014 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.485901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.486014 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.485939    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.486014 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.485970    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.486014 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.485996    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtk9p\" (UniqueName: \"kubernetes.io/projected/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-kube-api-access-vtk9p\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.587313 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.587279    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.587523 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.587327    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.587523 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.587346    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.587523 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.587372    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.587523 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.587397    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.587523 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.587423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtk9p\" (UniqueName: \"kubernetes.io/projected/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-kube-api-access-vtk9p\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.587850 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.587797    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.587906 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.587848    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.587906 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.587883    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.589773 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.589753    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.589836 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.589802    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.598768 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.598748    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtk9p\" (UniqueName: \"kubernetes.io/projected/f0886454-f00f-4d4a-9d7b-87ef2ea73ecd-kube-api-access-vtk9p\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw\" (UID: \"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.716064 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.716038    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:42.856211 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:42.856181    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"]
Jul 08 23:13:42.858244 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:13:42.858205    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0886454_f00f_4d4a_9d7b_87ef2ea73ecd.slice/crio-22d530f0dea14214fd8446a25246dcb6745647adc530b338d0e5222bad8945e5 WatchSource:0}: Error finding container 22d530f0dea14214fd8446a25246dcb6745647adc530b338d0e5222bad8945e5: Status 404 returned error can't find the container with id 22d530f0dea14214fd8446a25246dcb6745647adc530b338d0e5222bad8945e5
Jul 08 23:13:43.631505 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:43.631454    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw" event={"ID":"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd","Type":"ContainerStarted","Data":"22d530f0dea14214fd8446a25246dcb6745647adc530b338d0e5222bad8945e5"}
Jul 08 23:13:44.637599 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:44.637560    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw" event={"ID":"f0886454-f00f-4d4a-9d7b-87ef2ea73ecd","Type":"ContainerStarted","Data":"10021188f39908507c5de789df639e9d261ef80dc19d3b42e0a21960e31525bb"}
Jul 08 23:13:44.653315 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:44.653248    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw" podStartSLOduration=1.447734906 podStartE2EDuration="2.653232447s" totalImagesPullingTime="1.205497541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:13:44.652481206 +0000 UTC m=+643.582515171" watchObservedRunningTime="2026-07-08 23:13:44.653232447 +0000 UTC m=+643.583266420"
Jul 08 23:13:49.369778 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:49.369740    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:51.541706 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:51.541680    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:52.716307 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:52.716271    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:52.716761 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:52.716336    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:52.729747 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:52.729719    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:53.691331 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:53.691306    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw"
Jul 08 23:13:59.230347 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:59.230312    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:13:59.282598 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:13:59.282565    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:14:00.450589 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:00.450552    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:14:01.504751 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:01.504720    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:14:07.725105 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:07.725064    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:14:24.001496 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.001451    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"]
Jul 08 23:14:24.005155 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.005132    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.007655 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.007628    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 08 23:14:24.014480 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.014442    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"]
Jul 08 23:14:24.026301 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.026277    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:14:24.079231 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.079208    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.079364 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.079236    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34bf407c-e983-4019-bf42-fa9b342fd492-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.079364 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.079271    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.079364 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.079315    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.079364 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.079357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jk64\" (UniqueName: \"kubernetes.io/projected/34bf407c-e983-4019-bf42-fa9b342fd492-kube-api-access-6jk64\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.079529 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.079400    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.180901 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.180867    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.180901 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.180900    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34bf407c-e983-4019-bf42-fa9b342fd492-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.181121 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.180935    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.181121 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.180963    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.181121 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.181011    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jk64\" (UniqueName: \"kubernetes.io/projected/34bf407c-e983-4019-bf42-fa9b342fd492-kube-api-access-6jk64\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.181282 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.181143    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.181447 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.181407    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.181595 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.181460    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.181595 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.181518    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.183047 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.183029    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34bf407c-e983-4019-bf42-fa9b342fd492-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.183448 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.183428    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34bf407c-e983-4019-bf42-fa9b342fd492-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.189709 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.189680    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jk64\" (UniqueName: \"kubernetes.io/projected/34bf407c-e983-4019-bf42-fa9b342fd492-kube-api-access-6jk64\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-67hc9\" (UID: \"34bf407c-e983-4019-bf42-fa9b342fd492\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.315406 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.315326    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:24.448792 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.448729    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"]
Jul 08 23:14:24.451302 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:14:24.451272    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf407c_e983_4019_bf42_fa9b342fd492.slice/crio-2da2f8c71deac5dccaf8a8f7c0d5f7a11be265b7a5d4bd85318a70943f973a12 WatchSource:0}: Error finding container 2da2f8c71deac5dccaf8a8f7c0d5f7a11be265b7a5d4bd85318a70943f973a12: Status 404 returned error can't find the container with id 2da2f8c71deac5dccaf8a8f7c0d5f7a11be265b7a5d4bd85318a70943f973a12
Jul 08 23:14:24.818404 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.818367    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9" event={"ID":"34bf407c-e983-4019-bf42-fa9b342fd492","Type":"ContainerStarted","Data":"2dce28c3543e7fcdc7b8b5ae9c4f5876b46c46128b1f357e77a7967eb779a6c3"}
Jul 08 23:14:24.818600 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.818423    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9" event={"ID":"34bf407c-e983-4019-bf42-fa9b342fd492","Type":"ContainerStarted","Data":"2da2f8c71deac5dccaf8a8f7c0d5f7a11be265b7a5d4bd85318a70943f973a12"}
Jul 08 23:14:24.834597 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:24.834550    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9" podStartSLOduration=1.6532149459999999 podStartE2EDuration="1.834536288s" totalImagesPullingTime="181.321342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:14:24.833361025 +0000 UTC m=+683.763395001" watchObservedRunningTime="2026-07-08 23:14:24.834536288 +0000 UTC m=+683.764570313"
Jul 08 23:14:28.594403 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.594364    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"]
Jul 08 23:14:28.596715 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.596695    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.599029 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.599005    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 08 23:14:28.608683 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.608660    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"]
Jul 08 23:14:28.623320 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.623297    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.623443 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.623333    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.623443 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.623353    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d29c68-9624-40aa-84a5-9b68184aa2b0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.623443 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.623369    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.623443 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.623390    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjmjc\" (UniqueName: \"kubernetes.io/projected/a7d29c68-9624-40aa-84a5-9b68184aa2b0-kube-api-access-kjmjc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.623443 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.623432    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.723981 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.723936    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.723981 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.723991    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.724238 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.724013    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d29c68-9624-40aa-84a5-9b68184aa2b0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.724238 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.724038    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.724238 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.724060    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjmjc\" (UniqueName: \"kubernetes.io/projected/a7d29c68-9624-40aa-84a5-9b68184aa2b0-kube-api-access-kjmjc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.724238 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.724222    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.724457 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.724324    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.724599 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.724570    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.724725 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.724708    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.726322 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.726304    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7d29c68-9624-40aa-84a5-9b68184aa2b0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.726590 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.726573    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7d29c68-9624-40aa-84a5-9b68184aa2b0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.731499 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.731456    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjmjc\" (UniqueName: \"kubernetes.io/projected/a7d29c68-9624-40aa-84a5-9b68184aa2b0-kube-api-access-kjmjc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k\" (UID: \"a7d29c68-9624-40aa-84a5-9b68184aa2b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:28.739130 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.739102    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:14:28.908270 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:28.908190    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:29.037902 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:29.037879    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"]
Jul 08 23:14:29.040210 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:14:29.040183    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7d29c68_9624_40aa_84a5_9b68184aa2b0.slice/crio-4e17bf215c7a4f817623037b4d43355c28ce1aaccc7ff96d82c28704e8a3d378 WatchSource:0}: Error finding container 4e17bf215c7a4f817623037b4d43355c28ce1aaccc7ff96d82c28704e8a3d378: Status 404 returned error can't find the container with id 4e17bf215c7a4f817623037b4d43355c28ce1aaccc7ff96d82c28704e8a3d378
Jul 08 23:14:29.844612 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:29.844577    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k" event={"ID":"a7d29c68-9624-40aa-84a5-9b68184aa2b0","Type":"ContainerStarted","Data":"f734281888b8a2200da9231e8df8b44011813c1d80e5f2fb652b6cb0d3efb8ee"}
Jul 08 23:14:29.844612 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:29.844619    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k" event={"ID":"a7d29c68-9624-40aa-84a5-9b68184aa2b0","Type":"ContainerStarted","Data":"4e17bf215c7a4f817623037b4d43355c28ce1aaccc7ff96d82c28704e8a3d378"}
Jul 08 23:14:29.862412 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:29.862363    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k" podStartSLOduration=1.67330191 podStartE2EDuration="1.862347809s" totalImagesPullingTime="189.045899ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:14:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:14:29.860229187 +0000 UTC m=+688.790263162" watchObservedRunningTime="2026-07-08 23:14:29.862347809 +0000 UTC m=+688.792381783"
Jul 08 23:14:34.315967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:34.315929    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:34.316357 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:34.315992    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:34.328543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:34.328519    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:34.880272 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:34.880240    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-67hc9"
Jul 08 23:14:38.908996 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:38.908964    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:38.909567 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:38.909008    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:38.921149 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:38.921125    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:14:39.903020 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:14:39.902991    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k"
Jul 08 23:15:13.052093 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:13.052061    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f6f84675b-9jg9j"]
Jul 08 23:15:13.054377 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:13.054359    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f6f84675b-9jg9j"
Jul 08 23:15:13.062398 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:13.062373    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f6f84675b-9jg9j"]
Jul 08 23:15:13.131293 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:13.131257    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bfee96e5-6fc8-41fb-bef6-30a3b3fd876d-tls-cert\") pod \"authorino-f6f84675b-9jg9j\" (UID: \"bfee96e5-6fc8-41fb-bef6-30a3b3fd876d\") " pod="kuadrant-system/authorino-f6f84675b-9jg9j"
Jul 08 23:15:13.131454 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:13.131300    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqb2v\" (UniqueName: \"kubernetes.io/projected/bfee96e5-6fc8-41fb-bef6-30a3b3fd876d-kube-api-access-mqb2v\") pod \"authorino-f6f84675b-9jg9j\" (UID: \"bfee96e5-6fc8-41fb-bef6-30a3b3fd876d\") " pod="kuadrant-system/authorino-f6f84675b-9jg9j"
Jul 08 23:15:13.232717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:13.232683    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bfee96e5-6fc8-41fb-bef6-30a3b3fd876d-tls-cert\") pod \"authorino-f6f84675b-9jg9j\" (UID: \"bfee96e5-6fc8-41fb-bef6-30a3b3fd876d\") " pod="kuadrant-system/authorino-f6f84675b-9jg9j"
Jul 08 23:15:13.232884 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:13.232726    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqb2v\" (UniqueName: \"kubernetes.io/projected/bfee96e5-6fc8-41fb-bef6-30a3b3fd876d-kube-api-access-mqb2v\") pod \"authorino-f6f84675b-9jg9j\" (UID: \"bfee96e5-6fc8-41fb-bef6-30a3b3fd876d\") " pod="kuadrant-system/authorino-f6f84675b-9jg9j"
Jul 08 23:15:13.235023 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:13.235003    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bfee96e5-6fc8-41fb-bef6-30a3b3fd876d-tls-cert\") pod \"authorino-f6f84675b-9jg9j\" (UID: \"bfee96e5-6fc8-41fb-bef6-30a3b3fd876d\") " pod="kuadrant-system/authorino-f6f84675b-9jg9j"
Jul 08 23:15:13.240841 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:13.240822    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqb2v\" (UniqueName: \"kubernetes.io/projected/bfee96e5-6fc8-41fb-bef6-30a3b3fd876d-kube-api-access-mqb2v\") pod \"authorino-f6f84675b-9jg9j\" (UID: \"bfee96e5-6fc8-41fb-bef6-30a3b3fd876d\") " pod="kuadrant-system/authorino-f6f84675b-9jg9j"
Jul 08 23:15:13.363503 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:13.363409    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f6f84675b-9jg9j"
Jul 08 23:15:13.489582 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:13.489516    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f6f84675b-9jg9j"]
Jul 08 23:15:13.492017 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:15:13.491989    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfee96e5_6fc8_41fb_bef6_30a3b3fd876d.slice/crio-164b81c48849ef76c84512362cc52e5239507de0da49c71fdb6a6457ed8dff06 WatchSource:0}: Error finding container 164b81c48849ef76c84512362cc52e5239507de0da49c71fdb6a6457ed8dff06: Status 404 returned error can't find the container with id 164b81c48849ef76c84512362cc52e5239507de0da49c71fdb6a6457ed8dff06
Jul 08 23:15:14.053108 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.053066    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f6f84675b-9jg9j" event={"ID":"bfee96e5-6fc8-41fb-bef6-30a3b3fd876d","Type":"ContainerStarted","Data":"6ded3afdd060de0285a0ff65e08dc9fde14247ae011433da1652a4d7f053396f"}
Jul 08 23:15:14.053525 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.053120    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f6f84675b-9jg9j" event={"ID":"bfee96e5-6fc8-41fb-bef6-30a3b3fd876d","Type":"ContainerStarted","Data":"164b81c48849ef76c84512362cc52e5239507de0da49c71fdb6a6457ed8dff06"}
Jul 08 23:15:14.068568 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.068434    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-f6f84675b-9jg9j" podStartSLOduration=0.747663333 podStartE2EDuration="1.068418723s" totalImagesPullingTime="320.75539ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:15:14.068249898 +0000 UTC m=+732.998283885" watchObservedRunningTime="2026-07-08 23:15:14.068418723 +0000 UTC m=+732.998452696"
Jul 08 23:15:14.099676 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.099627    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-554c4b5df-sfsmg"]
Jul 08 23:15:14.099939 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.099911    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-554c4b5df-sfsmg" podUID="9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0" containerName="authorino" containerID="cri-o://f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e" gracePeriod=30
Jul 08 23:15:14.356931 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.356898    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-554c4b5df-sfsmg"
Jul 08 23:15:14.450482 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.450427    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9stgm\" (UniqueName: \"kubernetes.io/projected/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-kube-api-access-9stgm\") pod \"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0\" (UID: \"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0\") "
Jul 08 23:15:14.450649 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.450525    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-tls-cert\") pod \"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0\" (UID: \"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0\") "
Jul 08 23:15:14.452576 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.452546    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-kube-api-access-9stgm" (OuterVolumeSpecName: "kube-api-access-9stgm") pod "9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0" (UID: "9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0"). InnerVolumeSpecName "kube-api-access-9stgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:15:14.460950 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.460922    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0" (UID: "9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:15:14.552077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.552044    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9stgm\" (UniqueName: \"kubernetes.io/projected/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-kube-api-access-9stgm\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:15:14.552077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.552073    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0-tls-cert\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:15:14.593952 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.593802    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:15:14.690142 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:14.690087    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:15:15.059589 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.059548    2560 generic.go:356] "Generic (PLEG): container finished" podID="9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0" containerID="f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e" exitCode=0
Jul 08 23:15:15.059589 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.059586    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-554c4b5df-sfsmg"
Jul 08 23:15:15.060138 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.059630    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-554c4b5df-sfsmg" event={"ID":"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0","Type":"ContainerDied","Data":"f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e"}
Jul 08 23:15:15.060138 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.059685    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-554c4b5df-sfsmg" event={"ID":"9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0","Type":"ContainerDied","Data":"a81406b2fd030794f5d52e53b33afba52b08c96ec83d22c10467a814c7ee60de"}
Jul 08 23:15:15.060138 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.059709    2560 scope.go:117] "RemoveContainer" containerID="f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e"
Jul 08 23:15:15.069257 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.069242    2560 scope.go:117] "RemoveContainer" containerID="f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e"
Jul 08 23:15:15.069540 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:15:15.069520    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e\": container with ID starting with f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e not found: ID does not exist" containerID="f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e"
Jul 08 23:15:15.069601 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.069550    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e"} err="failed to get container status \"f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e\": rpc error: code = NotFound desc = could not find container \"f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e\": container with ID starting with f6625f3e1fa9aabbddd3704cea1c8818af1e564321edde8c96787de981db7e5e not found: ID does not exist"
Jul 08 23:15:15.088216 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.088189    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-554c4b5df-sfsmg"]
Jul 08 23:15:15.091727 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.091708    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-554c4b5df-sfsmg"]
Jul 08 23:15:15.705449 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.705414    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0" path="/var/lib/kubelet/pods/9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0/volumes"
Jul 08 23:15:15.766225 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.766184    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:15:15.844940 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:15.844904    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:15:19.974370 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:19.974340    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:15:20.075482 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:20.075439    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:15:52.034048 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:15:52.034014    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:16:25.768567 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:25.768533    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:16:25.877416 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:25.877385    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cdd595cbc-95pnv"]
Jul 08 23:16:25.877693 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:25.877664    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6cdd595cbc-95pnv" podUID="6b385482-03ed-4ea3-b0cf-cee7fb69b0b9" containerName="manager" containerID="cri-o://33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579" gracePeriod=30
Jul 08 23:16:26.123292 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.123264    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:16:26.202524 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.202488    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2v42\" (UniqueName: \"kubernetes.io/projected/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-kube-api-access-q2v42\") pod \"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9\" (UID: \"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9\") "
Jul 08 23:16:26.202688 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.202580    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-webhook-cert\") pod \"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9\" (UID: \"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9\") "
Jul 08 23:16:26.204614 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.204588    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6b385482-03ed-4ea3-b0cf-cee7fb69b0b9" (UID: "6b385482-03ed-4ea3-b0cf-cee7fb69b0b9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:16:26.204728 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.204658    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-kube-api-access-q2v42" (OuterVolumeSpecName: "kube-api-access-q2v42") pod "6b385482-03ed-4ea3-b0cf-cee7fb69b0b9" (UID: "6b385482-03ed-4ea3-b0cf-cee7fb69b0b9"). InnerVolumeSpecName "kube-api-access-q2v42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:16:26.304140 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.304060    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-webhook-cert\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:16:26.304140 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.304087    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2v42\" (UniqueName: \"kubernetes.io/projected/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9-kube-api-access-q2v42\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:16:26.404366 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.404326    2560 generic.go:356] "Generic (PLEG): container finished" podID="6b385482-03ed-4ea3-b0cf-cee7fb69b0b9" containerID="33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579" exitCode=0
Jul 08 23:16:26.404548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.404373    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cdd595cbc-95pnv"
Jul 08 23:16:26.404548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.404388    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cdd595cbc-95pnv" event={"ID":"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9","Type":"ContainerDied","Data":"33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579"}
Jul 08 23:16:26.404548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.404437    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cdd595cbc-95pnv" event={"ID":"6b385482-03ed-4ea3-b0cf-cee7fb69b0b9","Type":"ContainerDied","Data":"86206d2b29a3bd78fc1d677feb7f73182a7ea360d492760d2ddb877b2fff25ad"}
Jul 08 23:16:26.404548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.404462    2560 scope.go:117] "RemoveContainer" containerID="33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579"
Jul 08 23:16:26.413268 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.413249    2560 scope.go:117] "RemoveContainer" containerID="33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579"
Jul 08 23:16:26.413531 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:16:26.413512    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579\": container with ID starting with 33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579 not found: ID does not exist" containerID="33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579"
Jul 08 23:16:26.413596 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.413540    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579"} err="failed to get container status \"33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579\": rpc error: code = NotFound desc = could not find container \"33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579\": container with ID starting with 33017a0bfd882c61db27c2f272efac7df70548af4de667a6ee11953487040579 not found: ID does not exist"
Jul 08 23:16:26.426639 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.426610    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cdd595cbc-95pnv"]
Jul 08 23:16:26.431454 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:26.431432    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cdd595cbc-95pnv"]
Jul 08 23:16:27.046104 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.046058    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:16:27.158071 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.158037    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cdd595cbc-46p46"]
Jul 08 23:16:27.158431 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.158419    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0" containerName="authorino"
Jul 08 23:16:27.158525 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.158432    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0" containerName="authorino"
Jul 08 23:16:27.158525 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.158454    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b385482-03ed-4ea3-b0cf-cee7fb69b0b9" containerName="manager"
Jul 08 23:16:27.158525 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.158460    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b385482-03ed-4ea3-b0cf-cee7fb69b0b9" containerName="manager"
Jul 08 23:16:27.158678 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.158539    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b385482-03ed-4ea3-b0cf-cee7fb69b0b9" containerName="manager"
Jul 08 23:16:27.158678 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.158548    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a4f5535-cfe8-4bf4-86d7-a9ff44b156c0" containerName="authorino"
Jul 08 23:16:27.162525 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.162509    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cdd595cbc-46p46"
Jul 08 23:16:27.165790 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.165767    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 08 23:16:27.165790 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.165790    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 08 23:16:27.165985 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.165767    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n84h8\""
Jul 08 23:16:27.169117 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.169097    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cdd595cbc-46p46"]
Jul 08 23:16:27.210214 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.210181    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lt87\" (UniqueName: \"kubernetes.io/projected/0d846b17-e0e1-4a00-937b-f51fd5fdcbba-kube-api-access-7lt87\") pod \"maas-controller-6cdd595cbc-46p46\" (UID: \"0d846b17-e0e1-4a00-937b-f51fd5fdcbba\") " pod="opendatahub/maas-controller-6cdd595cbc-46p46"
Jul 08 23:16:27.210375 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.210301    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d846b17-e0e1-4a00-937b-f51fd5fdcbba-webhook-cert\") pod \"maas-controller-6cdd595cbc-46p46\" (UID: \"0d846b17-e0e1-4a00-937b-f51fd5fdcbba\") " pod="opendatahub/maas-controller-6cdd595cbc-46p46"
Jul 08 23:16:27.310983 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.310896    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d846b17-e0e1-4a00-937b-f51fd5fdcbba-webhook-cert\") pod \"maas-controller-6cdd595cbc-46p46\" (UID: \"0d846b17-e0e1-4a00-937b-f51fd5fdcbba\") " pod="opendatahub/maas-controller-6cdd595cbc-46p46"
Jul 08 23:16:27.310983 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.310960    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lt87\" (UniqueName: \"kubernetes.io/projected/0d846b17-e0e1-4a00-937b-f51fd5fdcbba-kube-api-access-7lt87\") pod \"maas-controller-6cdd595cbc-46p46\" (UID: \"0d846b17-e0e1-4a00-937b-f51fd5fdcbba\") " pod="opendatahub/maas-controller-6cdd595cbc-46p46"
Jul 08 23:16:27.313280 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.313255    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d846b17-e0e1-4a00-937b-f51fd5fdcbba-webhook-cert\") pod \"maas-controller-6cdd595cbc-46p46\" (UID: \"0d846b17-e0e1-4a00-937b-f51fd5fdcbba\") " pod="opendatahub/maas-controller-6cdd595cbc-46p46"
Jul 08 23:16:27.318319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.318297    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lt87\" (UniqueName: \"kubernetes.io/projected/0d846b17-e0e1-4a00-937b-f51fd5fdcbba-kube-api-access-7lt87\") pod \"maas-controller-6cdd595cbc-46p46\" (UID: \"0d846b17-e0e1-4a00-937b-f51fd5fdcbba\") " pod="opendatahub/maas-controller-6cdd595cbc-46p46"
Jul 08 23:16:27.473438 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.473404    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cdd595cbc-46p46"
Jul 08 23:16:27.603548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.603458    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cdd595cbc-46p46"]
Jul 08 23:16:27.605978 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:16:27.605950    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d846b17_e0e1_4a00_937b_f51fd5fdcbba.slice/crio-7917dd10337cb6182f881d4c1037d663eb1279572c59b52a636e2dc3cf918322 WatchSource:0}: Error finding container 7917dd10337cb6182f881d4c1037d663eb1279572c59b52a636e2dc3cf918322: Status 404 returned error can't find the container with id 7917dd10337cb6182f881d4c1037d663eb1279572c59b52a636e2dc3cf918322
Jul 08 23:16:27.706483 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:27.706447    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b385482-03ed-4ea3-b0cf-cee7fb69b0b9" path="/var/lib/kubelet/pods/6b385482-03ed-4ea3-b0cf-cee7fb69b0b9/volumes"
Jul 08 23:16:28.417314 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:28.417227    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cdd595cbc-46p46" event={"ID":"0d846b17-e0e1-4a00-937b-f51fd5fdcbba","Type":"ContainerStarted","Data":"70feba5ce6f4d48d9267fd8a98b3e62b2c8c78df2f78797bc90ecf4cb7883d67"}
Jul 08 23:16:28.417314 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:28.417281    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cdd595cbc-46p46" event={"ID":"0d846b17-e0e1-4a00-937b-f51fd5fdcbba","Type":"ContainerStarted","Data":"7917dd10337cb6182f881d4c1037d663eb1279572c59b52a636e2dc3cf918322"}
Jul 08 23:16:28.417314 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:28.417312    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cdd595cbc-46p46"
Jul 08 23:16:28.433296 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:28.433250    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cdd595cbc-46p46" podStartSLOduration=0.978250432 podStartE2EDuration="1.433235059s" totalImagesPullingTime="454.984627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:16:28.43315145 +0000 UTC m=+807.363185427" watchObservedRunningTime="2026-07-08 23:16:28.433235059 +0000 UTC m=+807.363269027"
Jul 08 23:16:39.428972 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:16:39.428935    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cdd595cbc-46p46"
Jul 08 23:17:09.626320 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:17:09.626273    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:17:09.742584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:17:09.742546    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:17:09.842752 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:17:09.842714    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:17:15.487798 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:17:15.487765    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:17:15.637086 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:17:15.637043    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:17:15.764746 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:17:15.764667    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:17:15.865784 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:17:15.865751    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:17:18.845297 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:17:18.845269    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:17:21.413588 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:17:21.413553    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:17:21.487506 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:17:21.487457    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:18:01.626445 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:18:01.626373    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:18:01.633695 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:18:01.633386    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:19:21.228344 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:19:21.228316    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:21:12.631995 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:21:12.631916    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:21:12.681637 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:21:12.681606    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:23:01.657732 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:23:01.657703    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:23:01.661941 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:23:01.661916    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:23:38.503378 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:23:38.503344    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:23:38.590213 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:23:38.590168    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:23:58.964536 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:23:58.964441    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:26:29.481972 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:26:29.481933    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:26:50.568510 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:26:50.568442    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:01.697024 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:01.696990    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:28:01.698526 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:01.698504    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:28:02.001505 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:02.001390    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:08.111477 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:08.111412    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:10.806059 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:10.806018    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:19.068655 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:19.068614    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:24.166570 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:24.166490    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:27.030896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:27.030860    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:27.128104 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:27.128074    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:27.459283 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:27.459236    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:33.083903 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:33.083859    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:33.141085 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:33.141054    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:33.278143 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:33.278087    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:33.375918 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:33.375831    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:34.466013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.465981    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:34.544680 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.544647    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"]
Jul 08 23:28:34.548053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.548023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:34.548207 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.548154    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:34.550939 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.550921    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fda58ddc-serving-cert\""
Jul 08 23:28:34.558518 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.558497    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"]
Jul 08 23:28:34.614168 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.614142    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:34.652416 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.652385    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvjqv\" (UniqueName: \"kubernetes.io/projected/41f971df-bb97-403a-8c44-cac4672fe051-kube-api-access-bvjqv\") pod \"maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:34.652569 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.652421    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f971df-bb97-403a-8c44-cac4672fe051-openshift-service-ca\") pod \"maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:34.652615 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.652577    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41f971df-bb97-403a-8c44-cac4672fe051-maas-api-tls\") pod \"maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:34.753809 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.753736    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvjqv\" (UniqueName: \"kubernetes.io/projected/41f971df-bb97-403a-8c44-cac4672fe051-kube-api-access-bvjqv\") pod \"maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:34.753809 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.753776    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f971df-bb97-403a-8c44-cac4672fe051-openshift-service-ca\") pod \"maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:34.754004 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.753844    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41f971df-bb97-403a-8c44-cac4672fe051-maas-api-tls\") pod \"maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:34.754568 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.754546    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f971df-bb97-403a-8c44-cac4672fe051-openshift-service-ca\") pod \"maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:34.755010 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.754979    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:34.756501 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.756478    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41f971df-bb97-403a-8c44-cac4672fe051-maas-api-tls\") pod \"maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:34.761346 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.761325    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvjqv\" (UniqueName: \"kubernetes.io/projected/41f971df-bb97-403a-8c44-cac4672fe051-kube-api-access-bvjqv\") pod \"maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:34.860950 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.860916    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:34.989774 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.989749    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"]
Jul 08 23:28:34.991957 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:28:34.991930    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f971df_bb97_403a_8c44_cac4672fe051.slice/crio-7af9b04505b4497ea8a1f14abb539f8b9558a6b00989650786eae96bd6fbbea1 WatchSource:0}: Error finding container 7af9b04505b4497ea8a1f14abb539f8b9558a6b00989650786eae96bd6fbbea1: Status 404 returned error can't find the container with id 7af9b04505b4497ea8a1f14abb539f8b9558a6b00989650786eae96bd6fbbea1
Jul 08 23:28:34.993318 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:34.993298    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 23:28:35.020595 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:35.020529    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:35.141518 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:35.141491    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:35.832943 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:35.832853    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5" event={"ID":"41f971df-bb97-403a-8c44-cac4672fe051","Type":"ContainerStarted","Data":"73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987"}
Jul 08 23:28:35.832943 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:35.832903    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5" event={"ID":"41f971df-bb97-403a-8c44-cac4672fe051","Type":"ContainerStarted","Data":"7af9b04505b4497ea8a1f14abb539f8b9558a6b00989650786eae96bd6fbbea1"}
Jul 08 23:28:35.833357 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:35.832966    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:35.849610 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:35.849564    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5" podStartSLOduration=1.358883538 podStartE2EDuration="1.849550653s" totalImagesPullingTime="490.667115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:28:35.847710068 +0000 UTC m=+1534.777744041" watchObservedRunningTime="2026-07-08 23:28:35.849550653 +0000 UTC m=+1534.779584626"
Jul 08 23:28:37.390203 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:37.390167    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:39.447599 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:39.447566    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:39.640969 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:39.640920    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:39.740428 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:39.740337    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:39.776577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:39.776545    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"]
Jul 08 23:28:39.776777 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:39.776752    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5" podUID="41f971df-bb97-403a-8c44-cac4672fe051" containerName="maas-api" containerID="cri-o://73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987" gracePeriod=30
Jul 08 23:28:39.843338 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:39.843309    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:40.020801 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.020780    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:40.100974 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.100943    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvjqv\" (UniqueName: \"kubernetes.io/projected/41f971df-bb97-403a-8c44-cac4672fe051-kube-api-access-bvjqv\") pod \"41f971df-bb97-403a-8c44-cac4672fe051\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") "
Jul 08 23:28:40.101138 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.100993    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f971df-bb97-403a-8c44-cac4672fe051-openshift-service-ca\") pod \"41f971df-bb97-403a-8c44-cac4672fe051\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") "
Jul 08 23:28:40.101138 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.101017    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41f971df-bb97-403a-8c44-cac4672fe051-maas-api-tls\") pod \"41f971df-bb97-403a-8c44-cac4672fe051\" (UID: \"41f971df-bb97-403a-8c44-cac4672fe051\") "
Jul 08 23:28:40.101353 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.101324    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41f971df-bb97-403a-8c44-cac4672fe051-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "41f971df-bb97-403a-8c44-cac4672fe051" (UID: "41f971df-bb97-403a-8c44-cac4672fe051"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:28:40.103027 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.103003    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41f971df-bb97-403a-8c44-cac4672fe051-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "41f971df-bb97-403a-8c44-cac4672fe051" (UID: "41f971df-bb97-403a-8c44-cac4672fe051"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:28:40.103128 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.103051    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41f971df-bb97-403a-8c44-cac4672fe051-kube-api-access-bvjqv" (OuterVolumeSpecName: "kube-api-access-bvjqv") pod "41f971df-bb97-403a-8c44-cac4672fe051" (UID: "41f971df-bb97-403a-8c44-cac4672fe051"). InnerVolumeSpecName "kube-api-access-bvjqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:28:40.202107 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.202071    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvjqv\" (UniqueName: \"kubernetes.io/projected/41f971df-bb97-403a-8c44-cac4672fe051-kube-api-access-bvjqv\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:28:40.202107 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.202104    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f971df-bb97-403a-8c44-cac4672fe051-openshift-service-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:28:40.202308 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.202117    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41f971df-bb97-403a-8c44-cac4672fe051-maas-api-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:28:40.856283 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.856247    2560 generic.go:356] "Generic (PLEG): container finished" podID="41f971df-bb97-403a-8c44-cac4672fe051" containerID="73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987" exitCode=0
Jul 08 23:28:40.856788 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.856297    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"
Jul 08 23:28:40.856788 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.856337    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5" event={"ID":"41f971df-bb97-403a-8c44-cac4672fe051","Type":"ContainerDied","Data":"73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987"}
Jul 08 23:28:40.856788 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.856396    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5" event={"ID":"41f971df-bb97-403a-8c44-cac4672fe051","Type":"ContainerDied","Data":"7af9b04505b4497ea8a1f14abb539f8b9558a6b00989650786eae96bd6fbbea1"}
Jul 08 23:28:40.856788 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.856429    2560 scope.go:117] "RemoveContainer" containerID="73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987"
Jul 08 23:28:40.865320 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.865302    2560 scope.go:117] "RemoveContainer" containerID="73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987"
Jul 08 23:28:40.865611 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:28:40.865593    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987\": container with ID starting with 73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987 not found: ID does not exist" containerID="73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987"
Jul 08 23:28:40.865665 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.865620    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987"} err="failed to get container status \"73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987\": rpc error: code = NotFound desc = could not find container \"73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987\": container with ID starting with 73ccb16fb36e30ba3cbe213dae56df635a91d9d9e5964bbc046e3c71d4609987 not found: ID does not exist"
Jul 08 23:28:40.879704 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.879679    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"]
Jul 08 23:28:40.885429 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:40.885409    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fda58ddc-5bb5b7f764-jv6h5"]
Jul 08 23:28:41.674635 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:41.674605    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:41.704252 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:41.704221    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41f971df-bb97-403a-8c44-cac4672fe051" path="/var/lib/kubelet/pods/41f971df-bb97-403a-8c44-cac4672fe051/volumes"
Jul 08 23:28:45.292147 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:45.292114    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:45.393168 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:45.393137    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:45.488818 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:45.488788    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:45.542300 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:45.542223    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:46.592733 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.592699    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:46.672145 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.672111    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"]
Jul 08 23:28:46.672526 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.672513    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41f971df-bb97-403a-8c44-cac4672fe051" containerName="maas-api"
Jul 08 23:28:46.672586 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.672528    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="41f971df-bb97-403a-8c44-cac4672fe051" containerName="maas-api"
Jul 08 23:28:46.672624 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.672596    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="41f971df-bb97-403a-8c44-cac4672fe051" containerName="maas-api"
Jul 08 23:28:46.674278 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.674234    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:46.676894 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.676877    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:46.679549 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.679529    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1c7e051c-serving-cert\""
Jul 08 23:28:46.686892 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.686874    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"]
Jul 08 23:28:46.742547 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.742518    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:46.755247 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.755226    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dfc9309-407b-4afb-ad20-793f3f0098da-openshift-service-ca\") pod \"maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:46.755343 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.755253    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-977dj\" (UniqueName: \"kubernetes.io/projected/3dfc9309-407b-4afb-ad20-793f3f0098da-kube-api-access-977dj\") pod \"maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:46.755343 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.755290    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dfc9309-407b-4afb-ad20-793f3f0098da-maas-api-tls\") pod \"maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:46.853554 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.853458    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:46.856584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.856553    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dfc9309-407b-4afb-ad20-793f3f0098da-openshift-service-ca\") pod \"maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:46.856706 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.856586    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-977dj\" (UniqueName: \"kubernetes.io/projected/3dfc9309-407b-4afb-ad20-793f3f0098da-kube-api-access-977dj\") pod \"maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:46.856706 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.856631    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dfc9309-407b-4afb-ad20-793f3f0098da-maas-api-tls\") pod \"maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:46.857176 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.857157    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dfc9309-407b-4afb-ad20-793f3f0098da-openshift-service-ca\") pod \"maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:46.858973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.858951    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dfc9309-407b-4afb-ad20-793f3f0098da-maas-api-tls\") pod \"maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:46.871232 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.871208    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-977dj\" (UniqueName: \"kubernetes.io/projected/3dfc9309-407b-4afb-ad20-793f3f0098da-kube-api-access-977dj\") pod \"maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:46.988192 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:46.988163    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:47.118638 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:47.118615    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"]
Jul 08 23:28:47.121318 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:28:47.121292    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dfc9309_407b_4afb_ad20_793f3f0098da.slice/crio-532afee0ef657c77248fb007fec8ac991d67a580d137a1c4b58ab21ea716b2c0 WatchSource:0}: Error finding container 532afee0ef657c77248fb007fec8ac991d67a580d137a1c4b58ab21ea716b2c0: Status 404 returned error can't find the container with id 532afee0ef657c77248fb007fec8ac991d67a580d137a1c4b58ab21ea716b2c0
Jul 08 23:28:47.269950 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:47.269916    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:47.370796 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:47.370703    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:47.890679 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:47.890643    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr" event={"ID":"3dfc9309-407b-4afb-ad20-793f3f0098da","Type":"ContainerStarted","Data":"71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91"}
Jul 08 23:28:47.891051 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:47.890699    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr" event={"ID":"3dfc9309-407b-4afb-ad20-793f3f0098da","Type":"ContainerStarted","Data":"532afee0ef657c77248fb007fec8ac991d67a580d137a1c4b58ab21ea716b2c0"}
Jul 08 23:28:47.891051 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:47.890748    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:47.906839 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:47.906790    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr" podStartSLOduration=1.5855883579999999 podStartE2EDuration="1.906776596s" totalImagesPullingTime="321.188238ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:28:47.90560156 +0000 UTC m=+1546.835635534" watchObservedRunningTime="2026-07-08 23:28:47.906776596 +0000 UTC m=+1546.836810594"
Jul 08 23:28:48.781413 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:48.781387    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:51.695958 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:51.695923    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:51.999459 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:51.999385    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"]
Jul 08 23:28:51.999663 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:51.999636    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr" podUID="3dfc9309-407b-4afb-ad20-793f3f0098da" containerName="maas-api" containerID="cri-o://71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91" gracePeriod=30
Jul 08 23:28:52.030972 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.030935    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:52.244413 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.244388    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:52.305425 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.305361    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-977dj\" (UniqueName: \"kubernetes.io/projected/3dfc9309-407b-4afb-ad20-793f3f0098da-kube-api-access-977dj\") pod \"3dfc9309-407b-4afb-ad20-793f3f0098da\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") "
Jul 08 23:28:52.305425 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.305399    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dfc9309-407b-4afb-ad20-793f3f0098da-openshift-service-ca\") pod \"3dfc9309-407b-4afb-ad20-793f3f0098da\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") "
Jul 08 23:28:52.305629 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.305515    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dfc9309-407b-4afb-ad20-793f3f0098da-maas-api-tls\") pod \"3dfc9309-407b-4afb-ad20-793f3f0098da\" (UID: \"3dfc9309-407b-4afb-ad20-793f3f0098da\") "
Jul 08 23:28:52.305768 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.305746    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dfc9309-407b-4afb-ad20-793f3f0098da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3dfc9309-407b-4afb-ad20-793f3f0098da" (UID: "3dfc9309-407b-4afb-ad20-793f3f0098da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:28:52.305832 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.305807    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dfc9309-407b-4afb-ad20-793f3f0098da-openshift-service-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:28:52.307415 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.307391    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dfc9309-407b-4afb-ad20-793f3f0098da-kube-api-access-977dj" (OuterVolumeSpecName: "kube-api-access-977dj") pod "3dfc9309-407b-4afb-ad20-793f3f0098da" (UID: "3dfc9309-407b-4afb-ad20-793f3f0098da"). InnerVolumeSpecName "kube-api-access-977dj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:28:52.307526 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.307420    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dfc9309-407b-4afb-ad20-793f3f0098da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3dfc9309-407b-4afb-ad20-793f3f0098da" (UID: "3dfc9309-407b-4afb-ad20-793f3f0098da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:28:52.406990 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.406962    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dfc9309-407b-4afb-ad20-793f3f0098da-maas-api-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:28:52.406990 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.406987    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-977dj\" (UniqueName: \"kubernetes.io/projected/3dfc9309-407b-4afb-ad20-793f3f0098da-kube-api-access-977dj\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:28:52.914580 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.914546    2560 generic.go:356] "Generic (PLEG): container finished" podID="3dfc9309-407b-4afb-ad20-793f3f0098da" containerID="71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91" exitCode=0
Jul 08 23:28:52.914965 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.914592    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"
Jul 08 23:28:52.914965 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.914633    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr" event={"ID":"3dfc9309-407b-4afb-ad20-793f3f0098da","Type":"ContainerDied","Data":"71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91"}
Jul 08 23:28:52.914965 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.914688    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr" event={"ID":"3dfc9309-407b-4afb-ad20-793f3f0098da","Type":"ContainerDied","Data":"532afee0ef657c77248fb007fec8ac991d67a580d137a1c4b58ab21ea716b2c0"}
Jul 08 23:28:52.914965 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.914712    2560 scope.go:117] "RemoveContainer" containerID="71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91"
Jul 08 23:28:52.924224 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.924208    2560 scope.go:117] "RemoveContainer" containerID="71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91"
Jul 08 23:28:52.924453 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:28:52.924436    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91\": container with ID starting with 71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91 not found: ID does not exist" containerID="71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91"
Jul 08 23:28:52.924528 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.924460    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91"} err="failed to get container status \"71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91\": rpc error: code = NotFound desc = could not find container \"71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91\": container with ID starting with 71c1842c5ccfed1c1f07ecd13203f993eb54fec9f71f40a118035274d0fe4f91 not found: ID does not exist"
Jul 08 23:28:52.934557 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.934535    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"]
Jul 08 23:28:52.938650 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:52.938629    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c7e051c-c96d958f4-kkwwr"]
Jul 08 23:28:53.704346 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:53.704312    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dfc9309-407b-4afb-ad20-793f3f0098da" path="/var/lib/kubelet/pods/3dfc9309-407b-4afb-ad20-793f3f0098da/volumes"
Jul 08 23:28:57.751025 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:57.750987    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:57.848154 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:57.848121    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:28:59.377255 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:28:59.377219    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:03.455877 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:03.455840    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:03.516530 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:03.516492    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:03.755900 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:03.755861    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:03.855105 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:03.855070    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:04.867576 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:04.867539    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:04.967619 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:04.967566    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:04.996683 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:04.996653    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"]
Jul 08 23:29:04.997045 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:04.997033    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dfc9309-407b-4afb-ad20-793f3f0098da" containerName="maas-api"
Jul 08 23:29:04.997093 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:04.997047    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dfc9309-407b-4afb-ad20-793f3f0098da" containerName="maas-api"
Jul 08 23:29:04.997127 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:04.997121    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dfc9309-407b-4afb-ad20-793f3f0098da" containerName="maas-api"
Jul 08 23:29:05.001121 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.001104    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.003493 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.003455    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-cb3326e4-serving-cert\""
Jul 08 23:29:05.011606 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.011586    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"]
Jul 08 23:29:05.086917 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.086888    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:05.117072 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.117041    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-maas-api-tls\") pod \"maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.117201 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.117092    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-openshift-service-ca\") pod \"maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.117201 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.117142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rkmd\" (UniqueName: \"kubernetes.io/projected/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-kube-api-access-8rkmd\") pod \"maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.189812 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.189780    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:05.218329 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.218304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-maas-api-tls\") pod \"maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.218462 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.218349    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-openshift-service-ca\") pod \"maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.218462 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.218372    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rkmd\" (UniqueName: \"kubernetes.io/projected/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-kube-api-access-8rkmd\") pod \"maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.218973 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.218950    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-openshift-service-ca\") pod \"maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.220707 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.220687    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-maas-api-tls\") pod \"maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.225817 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.225798    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rkmd\" (UniqueName: \"kubernetes.io/projected/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-kube-api-access-8rkmd\") pod \"maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.311699 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.311675    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.442302 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.442279    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"]
Jul 08 23:29:05.444307 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:29:05.444275    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb59b7d_a0a3_4380_bc80_09413cfab2f7.slice/crio-ccb417a3df2ab617c75e4da91900f951cae6adde48912f2c627bc59b82efe157 WatchSource:0}: Error finding container ccb417a3df2ab617c75e4da91900f951cae6adde48912f2c627bc59b82efe157: Status 404 returned error can't find the container with id ccb417a3df2ab617c75e4da91900f951cae6adde48912f2c627bc59b82efe157
Jul 08 23:29:05.515741 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.515706    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:05.622010 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.621976    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:05.976049 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.976013    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6" event={"ID":"7fb59b7d-a0a3-4380-bc80-09413cfab2f7","Type":"ContainerStarted","Data":"8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b"}
Jul 08 23:29:05.976460 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.976077    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6" event={"ID":"7fb59b7d-a0a3-4380-bc80-09413cfab2f7","Type":"ContainerStarted","Data":"ccb417a3df2ab617c75e4da91900f951cae6adde48912f2c627bc59b82efe157"}
Jul 08 23:29:05.976460 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.976105    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:05.992410 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:05.992318    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6" podStartSLOduration=1.6951004680000001 podStartE2EDuration="1.992300894s" totalImagesPullingTime="297.200426ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:29:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:29:05.990505625 +0000 UTC m=+1564.920539596" watchObservedRunningTime="2026-07-08 23:29:05.992300894 +0000 UTC m=+1564.922334867"
Jul 08 23:29:06.869106 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:06.869067    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:09.223195 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.223161    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:09.409139 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.409108    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:09.509038 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.508936    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:09.560330 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.560301    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"]
Jul 08 23:29:09.560577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.560545    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6" podUID="7fb59b7d-a0a3-4380-bc80-09413cfab2f7" containerName="maas-api" containerID="cri-o://8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b" gracePeriod=30
Jul 08 23:29:09.579958 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.579929    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:09.821788 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.821763    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:09.960793 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.960763    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-openshift-service-ca\") pod \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") "
Jul 08 23:29:09.960978 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.960814    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rkmd\" (UniqueName: \"kubernetes.io/projected/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-kube-api-access-8rkmd\") pod \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") "
Jul 08 23:29:09.960978 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.960834    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-maas-api-tls\") pod \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\" (UID: \"7fb59b7d-a0a3-4380-bc80-09413cfab2f7\") "
Jul 08 23:29:09.961107 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.961089    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7fb59b7d-a0a3-4380-bc80-09413cfab2f7" (UID: "7fb59b7d-a0a3-4380-bc80-09413cfab2f7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:29:09.962850 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.962827    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7fb59b7d-a0a3-4380-bc80-09413cfab2f7" (UID: "7fb59b7d-a0a3-4380-bc80-09413cfab2f7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:29:09.962967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.962938    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-kube-api-access-8rkmd" (OuterVolumeSpecName: "kube-api-access-8rkmd") pod "7fb59b7d-a0a3-4380-bc80-09413cfab2f7" (UID: "7fb59b7d-a0a3-4380-bc80-09413cfab2f7"). InnerVolumeSpecName "kube-api-access-8rkmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:29:09.995520 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.995489    2560 generic.go:356] "Generic (PLEG): container finished" podID="7fb59b7d-a0a3-4380-bc80-09413cfab2f7" containerID="8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b" exitCode=0
Jul 08 23:29:09.995646 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.995545    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"
Jul 08 23:29:09.995646 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.995578    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6" event={"ID":"7fb59b7d-a0a3-4380-bc80-09413cfab2f7","Type":"ContainerDied","Data":"8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b"}
Jul 08 23:29:09.995646 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.995637    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6" event={"ID":"7fb59b7d-a0a3-4380-bc80-09413cfab2f7","Type":"ContainerDied","Data":"ccb417a3df2ab617c75e4da91900f951cae6adde48912f2c627bc59b82efe157"}
Jul 08 23:29:09.995800 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:09.995661    2560 scope.go:117] "RemoveContainer" containerID="8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b"
Jul 08 23:29:10.004110 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:10.004085    2560 scope.go:117] "RemoveContainer" containerID="8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b"
Jul 08 23:29:10.004363 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:29:10.004346    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b\": container with ID starting with 8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b not found: ID does not exist" containerID="8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b"
Jul 08 23:29:10.004410 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:10.004368    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b"} err="failed to get container status \"8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b\": rpc error: code = NotFound desc = could not find container \"8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b\": container with ID starting with 8956d252a6948743ebd03674463e08c3eb838ad53a6b306aef94df2c79f84e1b not found: ID does not exist"
Jul 08 23:29:10.016035 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:10.016006    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"]
Jul 08 23:29:10.019210 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:10.019187    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cb3326e4-66869b8c6-zsgp6"]
Jul 08 23:29:10.062359 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:10.062289    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-openshift-service-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:29:10.062359 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:10.062316    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rkmd\" (UniqueName: \"kubernetes.io/projected/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-kube-api-access-8rkmd\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:29:10.062359 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:10.062329    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fb59b7d-a0a3-4380-bc80-09413cfab2f7-maas-api-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:29:11.704908 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:11.704865    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fb59b7d-a0a3-4380-bc80-09413cfab2f7" path="/var/lib/kubelet/pods/7fb59b7d-a0a3-4380-bc80-09413cfab2f7/volumes"
Jul 08 23:29:11.775669 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:11.775622    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:13.422716 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:13.422682    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:15.158641 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:15.158603    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:15.231609 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:15.231575    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:15.621496 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:15.621436    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:15.730361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:15.730327    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:15.824369 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:15.824338    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:17.142629 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:17.142592    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:22.964508 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:22.964443    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:23.068699 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:23.068663    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:23.164868 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:23.164832    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:23.264938 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:23.264848    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:23.638802 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:23.638724    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:49.586034 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:49.585996    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:55.192654 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:55.192563    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:55.293556 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:55.293516    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:29:55.392417 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:29:55.392377    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:11.258296 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:11.258258    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:17.013116 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:17.013080    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:17.161699 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:17.161665    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:17.277081 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:17.276992    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:27.741990 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:27.741953    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:28.497916 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:28.497876    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:34.192144 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:34.192108    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:34.296896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:34.296861    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:34.393700 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:34.393660    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:34.453554 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:34.453409    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:35.205078 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:35.205033    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:35.304070 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:35.304033    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:35.405059 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:35.405014    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:36.929852 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:36.929814    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:42.483447 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:42.483406    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:42.586518 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:42.586485    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:43.158238 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:43.158204    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:48.853979 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:48.853938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:48.953512 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:48.953481    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:49.094594 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:49.094541    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:49.388671 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:49.388638    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:50.037936 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:50.037899    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:52.135370 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.135322    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:52.211494 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.211453    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"]
Jul 08 23:30:52.211978 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.211961    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fb59b7d-a0a3-4380-bc80-09413cfab2f7" containerName="maas-api"
Jul 08 23:30:52.212030 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.211982    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fb59b7d-a0a3-4380-bc80-09413cfab2f7" containerName="maas-api"
Jul 08 23:30:52.212064 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.212055    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fb59b7d-a0a3-4380-bc80-09413cfab2f7" containerName="maas-api"
Jul 08 23:30:52.217375 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.217348    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:52.218757 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.218723    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:52.221808 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.221787    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-999d73b3-serving-cert\""
Jul 08 23:30:52.227605 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.227583    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"]
Jul 08 23:30:52.281308 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.281277    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b126f918-abc9-49ed-8815-54116c86397c-openshift-service-ca\") pod \"maas-api-e2e-mt-999d73b3-84676d9674-wtk7m\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:52.281460 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.281321    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b126f918-abc9-49ed-8815-54116c86397c-maas-api-tls\") pod \"maas-api-e2e-mt-999d73b3-84676d9674-wtk7m\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:52.281460 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.281352    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ptwx\" (UniqueName: \"kubernetes.io/projected/b126f918-abc9-49ed-8815-54116c86397c-kube-api-access-8ptwx\") pod \"maas-api-e2e-mt-999d73b3-84676d9674-wtk7m\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:52.286798 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.286767    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:52.382245 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.382210    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b126f918-abc9-49ed-8815-54116c86397c-openshift-service-ca\") pod \"maas-api-e2e-mt-999d73b3-84676d9674-wtk7m\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:52.382416 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.382251    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b126f918-abc9-49ed-8815-54116c86397c-maas-api-tls\") pod \"maas-api-e2e-mt-999d73b3-84676d9674-wtk7m\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:52.382416 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.382273    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ptwx\" (UniqueName: \"kubernetes.io/projected/b126f918-abc9-49ed-8815-54116c86397c-kube-api-access-8ptwx\") pod \"maas-api-e2e-mt-999d73b3-84676d9674-wtk7m\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:52.382416 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:30:52.382396    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-serving-cert: secret "maas-api-e2e-mt-999d73b3-serving-cert" not found
Jul 08 23:30:52.382552 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:30:52.382494    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b126f918-abc9-49ed-8815-54116c86397c-maas-api-tls podName:b126f918-abc9-49ed-8815-54116c86397c nodeName:}" failed. No retries permitted until 2026-07-08 23:30:52.882444805 +0000 UTC m=+1671.812478771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b126f918-abc9-49ed-8815-54116c86397c-maas-api-tls") pod "maas-api-e2e-mt-999d73b3-84676d9674-wtk7m" (UID: "b126f918-abc9-49ed-8815-54116c86397c") : secret "maas-api-e2e-mt-999d73b3-serving-cert" not found
Jul 08 23:30:52.382876 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.382856    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b126f918-abc9-49ed-8815-54116c86397c-openshift-service-ca\") pod \"maas-api-e2e-mt-999d73b3-84676d9674-wtk7m\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:52.391604 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.391546    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ptwx\" (UniqueName: \"kubernetes.io/projected/b126f918-abc9-49ed-8815-54116c86397c-kube-api-access-8ptwx\") pod \"maas-api-e2e-mt-999d73b3-84676d9674-wtk7m\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:52.444929 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.444891    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:52.798092 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.798059    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:52.888080 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.888040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b126f918-abc9-49ed-8815-54116c86397c-maas-api-tls\") pod \"maas-api-e2e-mt-999d73b3-84676d9674-wtk7m\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:52.890482 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.890444    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b126f918-abc9-49ed-8815-54116c86397c-maas-api-tls\") pod \"maas-api-e2e-mt-999d73b3-84676d9674-wtk7m\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:52.902037 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:52.902012    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:53.129252 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:53.129163    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:53.263539 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:53.263513    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"]
Jul 08 23:30:53.265841 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:30:53.265812    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb126f918_abc9_49ed_8815_54116c86397c.slice/crio-bf844c85cc4017ec90f1acb22a443d5c89164cbc91260db2359d686a2199d68c WatchSource:0}: Error finding container bf844c85cc4017ec90f1acb22a443d5c89164cbc91260db2359d686a2199d68c: Status 404 returned error can't find the container with id bf844c85cc4017ec90f1acb22a443d5c89164cbc91260db2359d686a2199d68c
Jul 08 23:30:53.468908 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:53.468876    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m" event={"ID":"b126f918-abc9-49ed-8815-54116c86397c","Type":"ContainerStarted","Data":"bf844c85cc4017ec90f1acb22a443d5c89164cbc91260db2359d686a2199d68c"}
Jul 08 23:30:54.474936 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:54.474901    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m" event={"ID":"b126f918-abc9-49ed-8815-54116c86397c","Type":"ContainerStarted","Data":"99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948"}
Jul 08 23:30:54.475334 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:54.475008    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:30:54.492338 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:54.492290    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m" podStartSLOduration=2.067225326 podStartE2EDuration="2.492275117s" totalImagesPullingTime="425.049791ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:30:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:30:54.490017899 +0000 UTC m=+1673.420051875" watchObservedRunningTime="2026-07-08 23:30:54.492275117 +0000 UTC m=+1673.422309090"
Jul 08 23:30:56.597953 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:56.597896    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:57.202758 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:57.202715    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:57.303280 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:57.303240    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:30:57.610446 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:30:57.610345    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:00.485411 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:00.485384    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:31:03.485907 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:03.485860    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:03.588727 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:03.588687    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:03.893146 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:03.893046    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:04.213438 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.213406    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"]
Jul 08 23:31:04.213679 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.213652    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m" podUID="b126f918-abc9-49ed-8815-54116c86397c" containerName="maas-api" containerID="cri-o://99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948" gracePeriod=30
Jul 08 23:31:04.243029 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.242967    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:04.303127 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.303092    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:04.463584 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.463523    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:31:04.522837 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.522804    2560 generic.go:356] "Generic (PLEG): container finished" podID="b126f918-abc9-49ed-8815-54116c86397c" containerID="99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948" exitCode=0
Jul 08 23:31:04.523217 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.522855    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"
Jul 08 23:31:04.523217 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.522884    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m" event={"ID":"b126f918-abc9-49ed-8815-54116c86397c","Type":"ContainerDied","Data":"99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948"}
Jul 08 23:31:04.523217 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.522939    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m" event={"ID":"b126f918-abc9-49ed-8815-54116c86397c","Type":"ContainerDied","Data":"bf844c85cc4017ec90f1acb22a443d5c89164cbc91260db2359d686a2199d68c"}
Jul 08 23:31:04.523217 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.522970    2560 scope.go:117] "RemoveContainer" containerID="99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948"
Jul 08 23:31:04.531058 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.531041    2560 scope.go:117] "RemoveContainer" containerID="99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948"
Jul 08 23:31:04.531334 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:31:04.531316    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948\": container with ID starting with 99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948 not found: ID does not exist" containerID="99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948"
Jul 08 23:31:04.531401 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.531343    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948"} err="failed to get container status \"99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948\": rpc error: code = NotFound desc = could not find container \"99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948\": container with ID starting with 99f18822166b5c940167fd48fb0514261971422f12d5f0e76200042a3c9b6948 not found: ID does not exist"
Jul 08 23:31:04.601072 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.601053    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b126f918-abc9-49ed-8815-54116c86397c-maas-api-tls\") pod \"b126f918-abc9-49ed-8815-54116c86397c\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") "
Jul 08 23:31:04.601169 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.601112    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ptwx\" (UniqueName: \"kubernetes.io/projected/b126f918-abc9-49ed-8815-54116c86397c-kube-api-access-8ptwx\") pod \"b126f918-abc9-49ed-8815-54116c86397c\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") "
Jul 08 23:31:04.601169 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.601145    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b126f918-abc9-49ed-8815-54116c86397c-openshift-service-ca\") pod \"b126f918-abc9-49ed-8815-54116c86397c\" (UID: \"b126f918-abc9-49ed-8815-54116c86397c\") "
Jul 08 23:31:04.601533 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.601502    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b126f918-abc9-49ed-8815-54116c86397c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b126f918-abc9-49ed-8815-54116c86397c" (UID: "b126f918-abc9-49ed-8815-54116c86397c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:31:04.603067 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.603047    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b126f918-abc9-49ed-8815-54116c86397c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b126f918-abc9-49ed-8815-54116c86397c" (UID: "b126f918-abc9-49ed-8815-54116c86397c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:31:04.603136 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.603082    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b126f918-abc9-49ed-8815-54116c86397c-kube-api-access-8ptwx" (OuterVolumeSpecName: "kube-api-access-8ptwx") pod "b126f918-abc9-49ed-8815-54116c86397c" (UID: "b126f918-abc9-49ed-8815-54116c86397c"). InnerVolumeSpecName "kube-api-access-8ptwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:31:04.702269 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.702236    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ptwx\" (UniqueName: \"kubernetes.io/projected/b126f918-abc9-49ed-8815-54116c86397c-kube-api-access-8ptwx\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:31:04.702269 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.702262    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b126f918-abc9-49ed-8815-54116c86397c-openshift-service-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:31:04.702269 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.702272    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b126f918-abc9-49ed-8815-54116c86397c-maas-api-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:31:04.843281 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.843247    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"]
Jul 08 23:31:04.846654 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:04.846632    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-999d73b3-84676d9674-wtk7m"]
Jul 08 23:31:05.704890 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:05.704854    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b126f918-abc9-49ed-8815-54116c86397c" path="/var/lib/kubelet/pods/b126f918-abc9-49ed-8815-54116c86397c/volumes"
Jul 08 23:31:09.976793 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:09.976757    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:10.043911 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:10.043872    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:10.217111 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:10.217079    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:10.551130 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:10.551092    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:10.651896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:10.651858    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:10.828612 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:10.828514    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:16.466112 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:16.466078    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:16.566748 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:16.566706    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:16.665930 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:16.665892    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:16.787583 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:16.787503    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:22.383437 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:22.383402    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:22.491490 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:22.491434    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:22.951684 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:22.951591    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:23.007008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:23.006971    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:29.125230 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:29.125193    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:29.433859 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:29.433826    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:29.647203 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:29.647162    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:32.358732 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.358679    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:32.429845 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.429815    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"]
Jul 08 23:31:32.430233 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.430208    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b126f918-abc9-49ed-8815-54116c86397c" containerName="maas-api"
Jul 08 23:31:32.430233 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.430224    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b126f918-abc9-49ed-8815-54116c86397c" containerName="maas-api"
Jul 08 23:31:32.430416 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.430308    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b126f918-abc9-49ed-8815-54116c86397c" containerName="maas-api"
Jul 08 23:31:32.433203 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.433184    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:32.435635 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.435606    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9ebdfa23-serving-cert\""
Jul 08 23:31:32.443915 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.443893    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"]
Jul 08 23:31:32.550929 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.550901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/992144ce-623a-428c-baa2-0846ece2ba4c-openshift-service-ca\") pod \"maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:32.551129 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.550949    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls\") pod \"maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:32.551129 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.550973    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlr29\" (UniqueName: \"kubernetes.io/projected/992144ce-623a-428c-baa2-0846ece2ba4c-kube-api-access-jlr29\") pod \"maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:32.652435 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.652354    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/992144ce-623a-428c-baa2-0846ece2ba4c-openshift-service-ca\") pod \"maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:32.652435 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.652403    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls\") pod \"maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:32.652645 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.652435    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlr29\" (UniqueName: \"kubernetes.io/projected/992144ce-623a-428c-baa2-0846ece2ba4c-kube-api-access-jlr29\") pod \"maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:32.652645 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:31:32.652573    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-serving-cert: secret "maas-api-e2e-mt-9ebdfa23-serving-cert" not found
Jul 08 23:31:32.652645 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:31:32.652641    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls podName:992144ce-623a-428c-baa2-0846ece2ba4c nodeName:}" failed. No retries permitted until 2026-07-08 23:31:33.15262073 +0000 UTC m=+1712.082654697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls") pod "maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9" (UID: "992144ce-623a-428c-baa2-0846ece2ba4c") : secret "maas-api-e2e-mt-9ebdfa23-serving-cert" not found
Jul 08 23:31:32.652985 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.652965    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/992144ce-623a-428c-baa2-0846ece2ba4c-openshift-service-ca\") pod \"maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:32.662161 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:32.662139    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlr29\" (UniqueName: \"kubernetes.io/projected/992144ce-623a-428c-baa2-0846ece2ba4c-kube-api-access-jlr29\") pod \"maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:33.157573 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:33.157536    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls\") pod \"maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:33.157757 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:31:33.157651    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-serving-cert: secret "maas-api-e2e-mt-9ebdfa23-serving-cert" not found
Jul 08 23:31:33.157757 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:31:33.157728    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls podName:992144ce-623a-428c-baa2-0846ece2ba4c nodeName:}" failed. No retries permitted until 2026-07-08 23:31:34.157712612 +0000 UTC m=+1713.087746564 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls") pod "maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9" (UID: "992144ce-623a-428c-baa2-0846ece2ba4c") : secret "maas-api-e2e-mt-9ebdfa23-serving-cert" not found
Jul 08 23:31:33.459325 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:33.459282    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:34.170148 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:34.170113    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls\") pod \"maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:34.172533 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:34.172509    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls\") pod \"maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:34.243954 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:34.243922    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:34.380712 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:34.380684    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"]
Jul 08 23:31:34.382523 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:31:34.382495    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod992144ce_623a_428c_baa2_0846ece2ba4c.slice/crio-95adbe794cb80a4ebd1511cd07ea68395f2ff93349b3123774cbcaa7397e6fde WatchSource:0}: Error finding container 95adbe794cb80a4ebd1511cd07ea68395f2ff93349b3123774cbcaa7397e6fde: Status 404 returned error can't find the container with id 95adbe794cb80a4ebd1511cd07ea68395f2ff93349b3123774cbcaa7397e6fde
Jul 08 23:31:34.669328 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:34.669240    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9" event={"ID":"992144ce-623a-428c-baa2-0846ece2ba4c","Type":"ContainerStarted","Data":"95adbe794cb80a4ebd1511cd07ea68395f2ff93349b3123774cbcaa7397e6fde"}
Jul 08 23:31:35.675538 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:35.675499    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9" event={"ID":"992144ce-623a-428c-baa2-0846ece2ba4c","Type":"ContainerStarted","Data":"732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96"}
Jul 08 23:31:35.676045 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:35.675638    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:35.694922 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:35.694876    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9" podStartSLOduration=3.278996296 podStartE2EDuration="3.694862545s" totalImagesPullingTime="415.866249ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:31:35.694330304 +0000 UTC m=+1714.624364284" watchObservedRunningTime="2026-07-08 23:31:35.694862545 +0000 UTC m=+1714.624896568"
Jul 08 23:31:36.869308 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:36.869268    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:41.685565 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:41.685532    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:31:42.093920 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.093834    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"]
Jul 08 23:31:42.097592 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.097569    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.099916 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.099894    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9ebdfa23\"/\"kube-root-ca.crt\""
Jul 08 23:31:42.100044 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.099940    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9ebdfa23\"/\"openshift-service-ca.crt\""
Jul 08 23:31:42.100817 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.100801    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9ebdfa23\"/\"default-dockercfg-dhb2f\""
Jul 08 23:31:42.100817 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.100807    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9ebdfa23\"/\"e2e-lifecycle-model-9ebdfa23-kserve-self-signed-certs\""
Jul 08 23:31:42.105730 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.105711    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"]
Jul 08 23:31:42.184261 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.184232    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:42.252382 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.252346    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-home\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.252577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.252413    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37138bc6-7c25-4146-9d72-efc7acb14a5b-tls-certs\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.252577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.252462    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-dshm\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.252577 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.252534    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6hb9\" (UniqueName: \"kubernetes.io/projected/37138bc6-7c25-4146-9d72-efc7acb14a5b-kube-api-access-l6hb9\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.252687 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.252578    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-tmp-dir\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.252687 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.252620    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-model-cache\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.353562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.353452    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37138bc6-7c25-4146-9d72-efc7acb14a5b-tls-certs\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.353562 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.353546    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-dshm\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.353759 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.353568    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6hb9\" (UniqueName: \"kubernetes.io/projected/37138bc6-7c25-4146-9d72-efc7acb14a5b-kube-api-access-l6hb9\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.353759 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.353588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-tmp-dir\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.353759 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.353630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-model-cache\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.353759 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.353679    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-home\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.354075 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.354052    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-tmp-dir\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.354167 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.354114    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-home\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.354167 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.354151    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-model-cache\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.355792 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.355763    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-dshm\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.355920 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.355903    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37138bc6-7c25-4146-9d72-efc7acb14a5b-tls-certs\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.361963 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.361939    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6hb9\" (UniqueName: \"kubernetes.io/projected/37138bc6-7c25-4146-9d72-efc7acb14a5b-kube-api-access-l6hb9\") pod \"e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") " pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.407666 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.407633    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:42.539539 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.539508    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"]
Jul 08 23:31:42.541578 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:31:42.541552    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37138bc6_7c25_4146_9d72_efc7acb14a5b.slice/crio-046ceb93c9fd6ec3391d9fbc3d3cd0e2694d987f9334104d13b9eaa3f54e9af4 WatchSource:0}: Error finding container 046ceb93c9fd6ec3391d9fbc3d3cd0e2694d987f9334104d13b9eaa3f54e9af4: Status 404 returned error can't find the container with id 046ceb93c9fd6ec3391d9fbc3d3cd0e2694d987f9334104d13b9eaa3f54e9af4
Jul 08 23:31:42.709378 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.709343    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq" event={"ID":"37138bc6-7c25-4146-9d72-efc7acb14a5b","Type":"ContainerStarted","Data":"bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195"}
Jul 08 23:31:42.709378 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.709383    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq" event={"ID":"37138bc6-7c25-4146-9d72-efc7acb14a5b","Type":"ContainerStarted","Data":"046ceb93c9fd6ec3391d9fbc3d3cd0e2694d987f9334104d13b9eaa3f54e9af4"}
Jul 08 23:31:42.743397 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:42.743349    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq" podStartSLOduration=0.743333815 podStartE2EDuration="743.333815ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:31:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:31:42.742342661 +0000 UTC m=+1721.672376647" watchObservedRunningTime="2026-07-08 23:31:42.743333815 +0000 UTC m=+1721.673367789"
Jul 08 23:31:49.521110 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:49.521075    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:49.622348 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:49.622311    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:52.408500 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:52.408459    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:52.408959 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:52.408530    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:52.421146 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:52.421120    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:52.765222 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:52.765192    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:31:53.712978 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:53.712931    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:31:53.812691 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:31:53.812639    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:32:09.159407 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.159369    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:32:09.479533 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.479502    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"]
Jul 08 23:32:09.479759 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.479732    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9" podUID="992144ce-623a-428c-baa2-0846ece2ba4c" containerName="maas-api" containerID="cri-o://732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96" gracePeriod=30
Jul 08 23:32:09.507607 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.507580    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:32:09.721987 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.721965    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:32:09.792983 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.792918    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlr29\" (UniqueName: \"kubernetes.io/projected/992144ce-623a-428c-baa2-0846ece2ba4c-kube-api-access-jlr29\") pod \"992144ce-623a-428c-baa2-0846ece2ba4c\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") "
Jul 08 23:32:09.792983 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.792964    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls\") pod \"992144ce-623a-428c-baa2-0846ece2ba4c\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") "
Jul 08 23:32:09.793202 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.792989    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/992144ce-623a-428c-baa2-0846ece2ba4c-openshift-service-ca\") pod \"992144ce-623a-428c-baa2-0846ece2ba4c\" (UID: \"992144ce-623a-428c-baa2-0846ece2ba4c\") "
Jul 08 23:32:09.793447 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.793412    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/992144ce-623a-428c-baa2-0846ece2ba4c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "992144ce-623a-428c-baa2-0846ece2ba4c" (UID: "992144ce-623a-428c-baa2-0846ece2ba4c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:32:09.794858 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.794833    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/992144ce-623a-428c-baa2-0846ece2ba4c-kube-api-access-jlr29" (OuterVolumeSpecName: "kube-api-access-jlr29") pod "992144ce-623a-428c-baa2-0846ece2ba4c" (UID: "992144ce-623a-428c-baa2-0846ece2ba4c"). InnerVolumeSpecName "kube-api-access-jlr29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:32:09.794962 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.794924    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "992144ce-623a-428c-baa2-0846ece2ba4c" (UID: "992144ce-623a-428c-baa2-0846ece2ba4c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:32:09.833034 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.833007    2560 generic.go:356] "Generic (PLEG): container finished" podID="992144ce-623a-428c-baa2-0846ece2ba4c" containerID="732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96" exitCode=0
Jul 08 23:32:09.833187 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.833040    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9" event={"ID":"992144ce-623a-428c-baa2-0846ece2ba4c","Type":"ContainerDied","Data":"732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96"}
Jul 08 23:32:09.833187 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.833044    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"
Jul 08 23:32:09.833187 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.833080    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9" event={"ID":"992144ce-623a-428c-baa2-0846ece2ba4c","Type":"ContainerDied","Data":"95adbe794cb80a4ebd1511cd07ea68395f2ff93349b3123774cbcaa7397e6fde"}
Jul 08 23:32:09.833187 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.833101    2560 scope.go:117] "RemoveContainer" containerID="732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96"
Jul 08 23:32:09.841733 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.841718    2560 scope.go:117] "RemoveContainer" containerID="732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96"
Jul 08 23:32:09.841990 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:32:09.841964    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96\": container with ID starting with 732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96 not found: ID does not exist" containerID="732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96"
Jul 08 23:32:09.842036 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.841989    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96"} err="failed to get container status \"732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96\": rpc error: code = NotFound desc = could not find container \"732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96\": container with ID starting with 732f52811a77ff3f2c792d961be54ae1d96df32bb343f5e7abadbad32052cb96 not found: ID does not exist"
Jul 08 23:32:09.853525 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.853505    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"]
Jul 08 23:32:09.858748 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.858729    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9ebdfa23-fdd66cb98-4lpj9"]
Jul 08 23:32:09.894673 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.894647    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlr29\" (UniqueName: \"kubernetes.io/projected/992144ce-623a-428c-baa2-0846ece2ba4c-kube-api-access-jlr29\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:32:09.894673 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.894666    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/992144ce-623a-428c-baa2-0846ece2ba4c-maas-api-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:32:09.894777 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:09.894676    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/992144ce-623a-428c-baa2-0846ece2ba4c-openshift-service-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:32:11.709415 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:11.709382    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="992144ce-623a-428c-baa2-0846ece2ba4c" path="/var/lib/kubelet/pods/992144ce-623a-428c-baa2-0846ece2ba4c/volumes"
Jul 08 23:32:15.177859 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:15.177827    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:32:20.161429 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:20.161397    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"]
Jul 08 23:32:20.203612 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:20.203566    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:32:35.181179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.181139    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq" podUID="37138bc6-7c25-4146-9d72-efc7acb14a5b" containerName="main" containerID="cri-o://bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195" gracePeriod=45
Jul 08 23:32:35.429518 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.429494    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:32:35.526618 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.526537    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-home\") pod \"37138bc6-7c25-4146-9d72-efc7acb14a5b\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") "
Jul 08 23:32:35.526618 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.526571    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-dshm\") pod \"37138bc6-7c25-4146-9d72-efc7acb14a5b\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") "
Jul 08 23:32:35.526618 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.526589    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-model-cache\") pod \"37138bc6-7c25-4146-9d72-efc7acb14a5b\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") "
Jul 08 23:32:35.526906 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.526631    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6hb9\" (UniqueName: \"kubernetes.io/projected/37138bc6-7c25-4146-9d72-efc7acb14a5b-kube-api-access-l6hb9\") pod \"37138bc6-7c25-4146-9d72-efc7acb14a5b\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") "
Jul 08 23:32:35.526906 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.526749    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37138bc6-7c25-4146-9d72-efc7acb14a5b-tls-certs\") pod \"37138bc6-7c25-4146-9d72-efc7acb14a5b\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") "
Jul 08 23:32:35.526906 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.526792    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-tmp-dir\") pod \"37138bc6-7c25-4146-9d72-efc7acb14a5b\" (UID: \"37138bc6-7c25-4146-9d72-efc7acb14a5b\") "
Jul 08 23:32:35.526906 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.526842    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-model-cache" (OuterVolumeSpecName: "model-cache") pod "37138bc6-7c25-4146-9d72-efc7acb14a5b" (UID: "37138bc6-7c25-4146-9d72-efc7acb14a5b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:32:35.526906 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.526857    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-home" (OuterVolumeSpecName: "home") pod "37138bc6-7c25-4146-9d72-efc7acb14a5b" (UID: "37138bc6-7c25-4146-9d72-efc7acb14a5b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:32:35.527179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.527048    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-home\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:32:35.527179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.527060    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-model-cache\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:32:35.527179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.527057    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "37138bc6-7c25-4146-9d72-efc7acb14a5b" (UID: "37138bc6-7c25-4146-9d72-efc7acb14a5b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:32:35.528850 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.528824    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-dshm" (OuterVolumeSpecName: "dshm") pod "37138bc6-7c25-4146-9d72-efc7acb14a5b" (UID: "37138bc6-7c25-4146-9d72-efc7acb14a5b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:32:35.528850 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.528841    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37138bc6-7c25-4146-9d72-efc7acb14a5b-kube-api-access-l6hb9" (OuterVolumeSpecName: "kube-api-access-l6hb9") pod "37138bc6-7c25-4146-9d72-efc7acb14a5b" (UID: "37138bc6-7c25-4146-9d72-efc7acb14a5b"). InnerVolumeSpecName "kube-api-access-l6hb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:32:35.528999 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.528909    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37138bc6-7c25-4146-9d72-efc7acb14a5b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "37138bc6-7c25-4146-9d72-efc7acb14a5b" (UID: "37138bc6-7c25-4146-9d72-efc7acb14a5b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:32:35.627783 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.627750    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-tmp-dir\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:32:35.627783 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.627776    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37138bc6-7c25-4146-9d72-efc7acb14a5b-dshm\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:32:35.627783 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.627786    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6hb9\" (UniqueName: \"kubernetes.io/projected/37138bc6-7c25-4146-9d72-efc7acb14a5b-kube-api-access-l6hb9\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:32:35.628013 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.627795    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37138bc6-7c25-4146-9d72-efc7acb14a5b-tls-certs\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:32:35.959462 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.959435    2560 generic.go:356] "Generic (PLEG): container finished" podID="37138bc6-7c25-4146-9d72-efc7acb14a5b" containerID="bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195" exitCode=0
Jul 08 23:32:35.959680 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.959499    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"
Jul 08 23:32:35.959680 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.959529    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq" event={"ID":"37138bc6-7c25-4146-9d72-efc7acb14a5b","Type":"ContainerDied","Data":"bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195"}
Jul 08 23:32:35.959680 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.959581    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq" event={"ID":"37138bc6-7c25-4146-9d72-efc7acb14a5b","Type":"ContainerDied","Data":"046ceb93c9fd6ec3391d9fbc3d3cd0e2694d987f9334104d13b9eaa3f54e9af4"}
Jul 08 23:32:35.959680 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.959611    2560 scope.go:117] "RemoveContainer" containerID="bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195"
Jul 08 23:32:35.968219 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.968201    2560 scope.go:117] "RemoveContainer" containerID="bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195"
Jul 08 23:32:35.968458 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:32:35.968438    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195\": container with ID starting with bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195 not found: ID does not exist" containerID="bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195"
Jul 08 23:32:35.968522 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.968481    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195"} err="failed to get container status \"bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195\": rpc error: code = NotFound desc = could not find container \"bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195\": container with ID starting with bdbb9dde46e4f45345d997209a35518017dfbbc5f50fc8e9af7d54dd05085195 not found: ID does not exist"
Jul 08 23:32:35.974915 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.974895    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"]
Jul 08 23:32:35.978678 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:35.978659    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-9ebdfa23/e2e-lifecycle-model-9ebdfa23-kserve-84c887d6bc-hbmwq"]
Jul 08 23:32:37.704090 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:37.704052    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37138bc6-7c25-4146-9d72-efc7acb14a5b" path="/var/lib/kubelet/pods/37138bc6-7c25-4146-9d72-efc7acb14a5b/volumes"
Jul 08 23:32:56.711975 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:56.711887    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:32:57.163256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:57.163174    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:32:59.651259 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:32:59.651186    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:01.732683 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:01.732652    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:33:01.733946 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:01.733927    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:33:02.602924 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:02.602885    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:02.703342 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:02.703308    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:02.870980 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:02.870894    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:04.127982 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:04.127947    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:04.222145 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:04.222106    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:04.328026 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:04.327959    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:05.032915 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:05.032881    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:05.127887 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:05.127843    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:05.227088 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:05.227027    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:06.575033 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:06.574994    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:12.399982 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:12.399951    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:12.454766 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:12.454733    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:13.044598 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:13.044568    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:18.783418 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:18.783376    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:18.851012 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:18.850957    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:18.956288 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:18.956260    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:19.143242 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:19.143165    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:29.492310 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:29.492271    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:29.821201 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:29.821126    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:50.757811 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:50.757760    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:56.446146 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:56.446115    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:56.546361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:56.546326    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:56.780909 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:56.780827    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:57.070707 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:57.070614    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:33:57.228557 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:33:57.228522    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:00.137545 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.137509    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:00.211064 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.211032    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"]
Jul 08 23:34:00.211542 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.211523    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37138bc6-7c25-4146-9d72-efc7acb14a5b" containerName="main"
Jul 08 23:34:00.211542 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.211542    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="37138bc6-7c25-4146-9d72-efc7acb14a5b" containerName="main"
Jul 08 23:34:00.211694 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.211555    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="992144ce-623a-428c-baa2-0846ece2ba4c" containerName="maas-api"
Jul 08 23:34:00.211694 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.211561    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="992144ce-623a-428c-baa2-0846ece2ba4c" containerName="maas-api"
Jul 08 23:34:00.211694 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.211644    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="992144ce-623a-428c-baa2-0846ece2ba4c" containerName="maas-api"
Jul 08 23:34:00.211694 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.211653    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="37138bc6-7c25-4146-9d72-efc7acb14a5b" containerName="main"
Jul 08 23:34:00.214029 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.213993    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:00.214964 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.214948    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:00.217354 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.217337    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-96223d-serving-cert\""
Jul 08 23:34:00.225648 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.225628    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"]
Jul 08 23:34:00.282311 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.282286    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:00.336583 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.336560    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9bfca4b-d0ec-4ffb-8100-729e3198964f-maas-api-tls\") pod \"maas-api-e2e-inf-a-96223d-79769cc688-drt5m\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:00.336685 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.336612    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b56pk\" (UniqueName: \"kubernetes.io/projected/d9bfca4b-d0ec-4ffb-8100-729e3198964f-kube-api-access-b56pk\") pod \"maas-api-e2e-inf-a-96223d-79769cc688-drt5m\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:00.336685 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.336665    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9bfca4b-d0ec-4ffb-8100-729e3198964f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-96223d-79769cc688-drt5m\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:00.382740 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.382713    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:00.437773 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.437751    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9bfca4b-d0ec-4ffb-8100-729e3198964f-maas-api-tls\") pod \"maas-api-e2e-inf-a-96223d-79769cc688-drt5m\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:00.437936 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.437800    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b56pk\" (UniqueName: \"kubernetes.io/projected/d9bfca4b-d0ec-4ffb-8100-729e3198964f-kube-api-access-b56pk\") pod \"maas-api-e2e-inf-a-96223d-79769cc688-drt5m\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:00.437936 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.437832    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9bfca4b-d0ec-4ffb-8100-729e3198964f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-96223d-79769cc688-drt5m\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:00.438427 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.438410    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9bfca4b-d0ec-4ffb-8100-729e3198964f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-96223d-79769cc688-drt5m\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:00.440227 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.440208    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9bfca4b-d0ec-4ffb-8100-729e3198964f-maas-api-tls\") pod \"maas-api-e2e-inf-a-96223d-79769cc688-drt5m\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:00.445827 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.445805    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b56pk\" (UniqueName: \"kubernetes.io/projected/d9bfca4b-d0ec-4ffb-8100-729e3198964f-kube-api-access-b56pk\") pod \"maas-api-e2e-inf-a-96223d-79769cc688-drt5m\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:00.526190 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.526171    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:00.683834 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.683811    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"]
Jul 08 23:34:00.685560 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:34:00.685531    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9bfca4b_d0ec_4ffb_8100_729e3198964f.slice/crio-27f2560307dcbc2841cdc7ee09d6f64e9c55b2e83f3065ea79f971d66ac0a819 WatchSource:0}: Error finding container 27f2560307dcbc2841cdc7ee09d6f64e9c55b2e83f3065ea79f971d66ac0a819: Status 404 returned error can't find the container with id 27f2560307dcbc2841cdc7ee09d6f64e9c55b2e83f3065ea79f971d66ac0a819
Jul 08 23:34:00.686878 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.686861    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 23:34:00.788662 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.788622    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:00.881064 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:00.881019    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:01.350847 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:01.350814    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m" event={"ID":"d9bfca4b-d0ec-4ffb-8100-729e3198964f","Type":"ContainerStarted","Data":"46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32"}
Jul 08 23:34:01.351174 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:01.350873    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:01.351174 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:01.350893    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m" event={"ID":"d9bfca4b-d0ec-4ffb-8100-729e3198964f","Type":"ContainerStarted","Data":"27f2560307dcbc2841cdc7ee09d6f64e9c55b2e83f3065ea79f971d66ac0a819"}
Jul 08 23:34:01.368244 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:01.368193    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m" podStartSLOduration=0.773804085 podStartE2EDuration="1.368179244s" totalImagesPullingTime="594.375159ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:34:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:34:01.365721459 +0000 UTC m=+1860.295755433" watchObservedRunningTime="2026-07-08 23:34:01.368179244 +0000 UTC m=+1860.298213218"
Jul 08 23:34:04.617242 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:04.617205    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:04.926527 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:04.926491    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:05.224098 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:05.224021    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:05.366050 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:05.366021    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:07.360895 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:07.360864    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:08.280497 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.280429    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:08.349895 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.349857    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"]
Jul 08 23:34:08.353519 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.353496    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:08.356158 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.356138    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-e9d26a-serving-cert\""
Jul 08 23:34:08.364280 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.364256    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"]
Jul 08 23:34:08.367242 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.367212    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:08.439127 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.439094    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:08.507217 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.507174    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:08.507407 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.507226    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrf9p\" (UniqueName: \"kubernetes.io/projected/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-kube-api-access-hrf9p\") pod \"maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:08.507407 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.507272    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-maas-api-tls\") pod \"maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:08.608189 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.608097    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:08.608189 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.608156    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrf9p\" (UniqueName: \"kubernetes.io/projected/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-kube-api-access-hrf9p\") pod \"maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:08.608189 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.608190    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-maas-api-tls\") pod \"maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:08.608444 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:34:08.608379    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-serving-cert: secret "maas-api-e2e-inf-b-e9d26a-serving-cert" not found
Jul 08 23:34:08.608444 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:34:08.608439    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-maas-api-tls podName:4d103efd-2ed2-4c89-b723-c7a07c0a5ee2 nodeName:}" failed. No retries permitted until 2026-07-08 23:34:09.108422897 +0000 UTC m=+1868.038456852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-maas-api-tls") pod "maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg" (UID: "4d103efd-2ed2-4c89-b723-c7a07c0a5ee2") : secret "maas-api-e2e-inf-b-e9d26a-serving-cert" not found
Jul 08 23:34:08.608811 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.608792    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:08.618897 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.618877    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrf9p\" (UniqueName: \"kubernetes.io/projected/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-kube-api-access-hrf9p\") pod \"maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:08.812556 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.812518    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:08.940810 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:08.940769    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:09.043900 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:09.043854    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:09.112962 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:09.112927    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-maas-api-tls\") pod \"maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:09.115209 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:09.115190    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-maas-api-tls\") pod \"maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:09.267905 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:09.267820    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:09.400622 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:09.400594    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"]
Jul 08 23:34:09.403216 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:34:09.403183    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d103efd_2ed2_4c89_b723_c7a07c0a5ee2.slice/crio-f1d5b58b63486af0b3b268e79096c950ea5dd392bba0e958c6b2ae8538e38fc0 WatchSource:0}: Error finding container f1d5b58b63486af0b3b268e79096c950ea5dd392bba0e958c6b2ae8538e38fc0: Status 404 returned error can't find the container with id f1d5b58b63486af0b3b268e79096c950ea5dd392bba0e958c6b2ae8538e38fc0
Jul 08 23:34:10.393654 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:10.393618    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg" event={"ID":"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2","Type":"ContainerStarted","Data":"5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2"}
Jul 08 23:34:10.393834 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:10.393678    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg" event={"ID":"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2","Type":"ContainerStarted","Data":"f1d5b58b63486af0b3b268e79096c950ea5dd392bba0e958c6b2ae8538e38fc0"}
Jul 08 23:34:10.393834 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:10.393756    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:10.416195 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:10.416152    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg" podStartSLOduration=2.064993275 podStartE2EDuration="2.416139484s" totalImagesPullingTime="351.146209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:34:10.414591179 +0000 UTC m=+1869.344625157" watchObservedRunningTime="2026-07-08 23:34:10.416139484 +0000 UTC m=+1869.346173458"
Jul 08 23:34:12.679181 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:12.679150    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:16.403357 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:16.403330    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:34:21.741991 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:21.741959    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:21.843067 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:21.843036    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:21.898923 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:21.898894    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"]
Jul 08 23:34:21.902411 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:21.902394    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:21.904695 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:21.904659    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-96223d\"/\"default-dockercfg-7fv6n\""
Jul 08 23:34:21.905757 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:21.905717    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-96223d\"/\"openshift-service-ca.crt\""
Jul 08 23:34:21.905757 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:21.905733    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-96223d\"/\"test-model-96223d-kserve-self-signed-certs\""
Jul 08 23:34:21.905757 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:21.905733    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-96223d\"/\"kube-root-ca.crt\""
Jul 08 23:34:21.910517 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:21.910494    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"]
Jul 08 23:34:21.923724 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:21.923699    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:22.033811 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.033714    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-home\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.033811 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.033752    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-dshm\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.033811 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.033788    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tls-certs\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.034112 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.033865    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-model-cache\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.034112 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.033949    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tmp-dir\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.034112 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.034061    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvg6b\" (UniqueName: \"kubernetes.io/projected/41a3a20b-ba1a-4e98-838e-0c1f728289b9-kube-api-access-wvg6b\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.135284 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.135255    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvg6b\" (UniqueName: \"kubernetes.io/projected/41a3a20b-ba1a-4e98-838e-0c1f728289b9-kube-api-access-wvg6b\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.135284 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.135291    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-home\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.135547 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.135309    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-dshm\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.135547 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.135416    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tls-certs\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.135547 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.135447    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-model-cache\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.135547 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.135534    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tmp-dir\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.135829 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.135806    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-home\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.135900 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.135851    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-model-cache\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.135940 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.135906    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tmp-dir\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.137707 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.137685    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-dshm\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.137984 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.137966    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tls-certs\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.142454 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.142435    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvg6b\" (UniqueName: \"kubernetes.io/projected/41a3a20b-ba1a-4e98-838e-0c1f728289b9-kube-api-access-wvg6b\") pod \"test-model-96223d-kserve-65794d6549-xc5qz\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") " pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.212643 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.212600    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:22.344390 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.344360    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"]
Jul 08 23:34:22.346732 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:34:22.346702    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41a3a20b_ba1a_4e98_838e_0c1f728289b9.slice/crio-b1460e92d3a76f243f214df85c80af3234e7eb6da67783b2130c713d1d9658a2 WatchSource:0}: Error finding container b1460e92d3a76f243f214df85c80af3234e7eb6da67783b2130c713d1d9658a2: Status 404 returned error can't find the container with id b1460e92d3a76f243f214df85c80af3234e7eb6da67783b2130c713d1d9658a2
Jul 08 23:34:22.453655 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.453615    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz" event={"ID":"41a3a20b-ba1a-4e98-838e-0c1f728289b9","Type":"ContainerStarted","Data":"d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7"}
Jul 08 23:34:22.453824 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.453665    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz" event={"ID":"41a3a20b-ba1a-4e98-838e-0c1f728289b9","Type":"ContainerStarted","Data":"b1460e92d3a76f243f214df85c80af3234e7eb6da67783b2130c713d1d9658a2"}
Jul 08 23:34:22.470552 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:22.470504    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz" podStartSLOduration=1.4704882320000001 podStartE2EDuration="1.470488232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:34:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:34:22.469406629 +0000 UTC m=+1881.399440605" watchObservedRunningTime="2026-07-08 23:34:22.470488232 +0000 UTC m=+1881.400522199"
Jul 08 23:34:29.675756 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:29.675718    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:29.774900 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:29.774864    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:32.213476 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:32.213426    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:32.213940 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:32.213575    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:32.226608 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:32.226580    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:32.513094 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:32.513007    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:34:37.195270 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.195231    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"]
Jul 08 23:34:37.198652 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.198630    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.200923 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.200895    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e9d26a\"/\"openshift-service-ca.crt\""
Jul 08 23:34:37.201054 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.200970    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e9d26a\"/\"test-model-e9d26a-kserve-self-signed-certs\""
Jul 08 23:34:37.201886 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.201870    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e9d26a\"/\"kube-root-ca.crt\""
Jul 08 23:34:37.201967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.201937    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e9d26a\"/\"default-dockercfg-zdvqd\""
Jul 08 23:34:37.206392 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.206365    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"]
Jul 08 23:34:37.214548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.214514    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:37.273905 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.273871    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-tmp-dir\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.274068 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.273940    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-dshm\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.274068 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.273970    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67ceed24-8045-4fa9-abd1-2f840a335662-tls-certs\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.274068 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.273990    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-model-cache\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.274172 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.274071    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-home\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.274172 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.274121    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbvsb\" (UniqueName: \"kubernetes.io/projected/67ceed24-8045-4fa9-abd1-2f840a335662-kube-api-access-bbvsb\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.375319 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.375283    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-tmp-dir\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.375523 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.375341    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-dshm\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.375523 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.375366    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67ceed24-8045-4fa9-abd1-2f840a335662-tls-certs\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.375523 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.375385    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-model-cache\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.375523 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.375409    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-home\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.375523 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.375494    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbvsb\" (UniqueName: \"kubernetes.io/projected/67ceed24-8045-4fa9-abd1-2f840a335662-kube-api-access-bbvsb\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.375802 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.375762    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-tmp-dir\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.375872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.375809    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-model-cache\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.375930 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.375902    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-home\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.377671 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.377651    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-dshm\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.377918 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.377900    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67ceed24-8045-4fa9-abd1-2f840a335662-tls-certs\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.384612 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.384586    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbvsb\" (UniqueName: \"kubernetes.io/projected/67ceed24-8045-4fa9-abd1-2f840a335662-kube-api-access-bbvsb\") pod \"test-model-e9d26a-kserve-ddb9c499f-cgb9q\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") " pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.509987 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.509888    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:37.645764 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:37.645738    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"]
Jul 08 23:34:37.648055 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:34:37.648030    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67ceed24_8045_4fa9_abd1_2f840a335662.slice/crio-e0db32920139f119d03093a421a7d2206767b9d07dc60ddd926648bc0358fdec WatchSource:0}: Error finding container e0db32920139f119d03093a421a7d2206767b9d07dc60ddd926648bc0358fdec: Status 404 returned error can't find the container with id e0db32920139f119d03093a421a7d2206767b9d07dc60ddd926648bc0358fdec
Jul 08 23:34:38.530431 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:38.530395    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q" event={"ID":"67ceed24-8045-4fa9-abd1-2f840a335662","Type":"ContainerStarted","Data":"3ae0674b6928327597803fef1e25279eddc559667bf078ae8006427dd0b62c28"}
Jul 08 23:34:38.530431 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:38.530436    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q" event={"ID":"67ceed24-8045-4fa9-abd1-2f840a335662","Type":"ContainerStarted","Data":"e0db32920139f119d03093a421a7d2206767b9d07dc60ddd926648bc0358fdec"}
Jul 08 23:34:38.547480 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:38.547424    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q" podStartSLOduration=1.54741007 podStartE2EDuration="1.54741007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:34:38.545180444 +0000 UTC m=+1897.475214419" watchObservedRunningTime="2026-07-08 23:34:38.54741007 +0000 UTC m=+1897.477444044"
Jul 08 23:34:47.510450 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:47.510414    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:47.510954 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:47.510492    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:47.522845 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:47.522813    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:47.585599 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:47.585573    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:34:55.577187 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:55.577154    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"]
Jul 08 23:34:55.599801 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:55.599770    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:56.427935 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:56.427905    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"]
Jul 08 23:34:56.448959 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:56.448925    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:56.627552 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:56.627494    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:56.996780 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:56.996746    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:57.098002 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.097957    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:57.197282 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.197238    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:57.313991 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.313890    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"]
Jul 08 23:34:57.314231 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.314201    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m" podUID="d9bfca4b-d0ec-4ffb-8100-729e3198964f" containerName="maas-api" containerID="cri-o://46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32" gracePeriod=30
Jul 08 23:34:57.336985 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.336955    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:34:57.356184 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.356148    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m" podUID="d9bfca4b-d0ec-4ffb-8100-729e3198964f" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.48:8443/health\": dial tcp 10.134.0.48:8443: connect: connection refused"
Jul 08 23:34:57.558456 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.558431    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:57.625437 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.625360    2560 generic.go:356] "Generic (PLEG): container finished" podID="d9bfca4b-d0ec-4ffb-8100-729e3198964f" containerID="46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32" exitCode=0
Jul 08 23:34:57.625437 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.625405    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"
Jul 08 23:34:57.625624 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.625402    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m" event={"ID":"d9bfca4b-d0ec-4ffb-8100-729e3198964f","Type":"ContainerDied","Data":"46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32"}
Jul 08 23:34:57.625624 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.625522    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m" event={"ID":"d9bfca4b-d0ec-4ffb-8100-729e3198964f","Type":"ContainerDied","Data":"27f2560307dcbc2841cdc7ee09d6f64e9c55b2e83f3065ea79f971d66ac0a819"}
Jul 08 23:34:57.625624 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.625554    2560 scope.go:117] "RemoveContainer" containerID="46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32"
Jul 08 23:34:57.634080 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.634062    2560 scope.go:117] "RemoveContainer" containerID="46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32"
Jul 08 23:34:57.634344 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:34:57.634304    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32\": container with ID starting with 46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32 not found: ID does not exist" containerID="46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32"
Jul 08 23:34:57.634344 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.634327    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32"} err="failed to get container status \"46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32\": rpc error: code = NotFound desc = could not find container \"46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32\": container with ID starting with 46ededf213e8f438a03b583b12e3d7e23a94f4a2793cd1495e8030c9e67e4e32 not found: ID does not exist"
Jul 08 23:34:57.655756 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.655734    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b56pk\" (UniqueName: \"kubernetes.io/projected/d9bfca4b-d0ec-4ffb-8100-729e3198964f-kube-api-access-b56pk\") pod \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") "
Jul 08 23:34:57.655845 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.655767    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9bfca4b-d0ec-4ffb-8100-729e3198964f-maas-api-tls\") pod \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") "
Jul 08 23:34:57.655845 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.655802    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9bfca4b-d0ec-4ffb-8100-729e3198964f-openshift-service-ca\") pod \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\" (UID: \"d9bfca4b-d0ec-4ffb-8100-729e3198964f\") "
Jul 08 23:34:57.656116 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.656095    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9bfca4b-d0ec-4ffb-8100-729e3198964f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9bfca4b-d0ec-4ffb-8100-729e3198964f" (UID: "d9bfca4b-d0ec-4ffb-8100-729e3198964f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:34:57.657792 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.657773    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9bfca4b-d0ec-4ffb-8100-729e3198964f-kube-api-access-b56pk" (OuterVolumeSpecName: "kube-api-access-b56pk") pod "d9bfca4b-d0ec-4ffb-8100-729e3198964f" (UID: "d9bfca4b-d0ec-4ffb-8100-729e3198964f"). InnerVolumeSpecName "kube-api-access-b56pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:34:57.657857 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.657775    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9bfca4b-d0ec-4ffb-8100-729e3198964f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9bfca4b-d0ec-4ffb-8100-729e3198964f" (UID: "d9bfca4b-d0ec-4ffb-8100-729e3198964f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:34:57.756924 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.756899    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b56pk\" (UniqueName: \"kubernetes.io/projected/d9bfca4b-d0ec-4ffb-8100-729e3198964f-kube-api-access-b56pk\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:34:57.756924 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.756922    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9bfca4b-d0ec-4ffb-8100-729e3198964f-maas-api-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:34:57.757059 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.756931    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9bfca4b-d0ec-4ffb-8100-729e3198964f-openshift-service-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:34:57.941239 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.941207    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"]
Jul 08 23:34:57.944413 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:57.944385    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96223d-79769cc688-drt5m"]
Jul 08 23:34:59.704374 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:34:59.704339    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9bfca4b-d0ec-4ffb-8100-729e3198964f" path="/var/lib/kubelet/pods/d9bfca4b-d0ec-4ffb-8100-729e3198964f/volumes"
Jul 08 23:35:02.072020 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:02.071938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:10.597510 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.597447    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz" podUID="41a3a20b-ba1a-4e98-838e-0c1f728289b9" containerName="main" containerID="cri-o://d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7" gracePeriod=45
Jul 08 23:35:10.850316 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.850257    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:35:10.878954 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.878932    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-dshm\") pod \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") "
Jul 08 23:35:10.879078 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.878963    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvg6b\" (UniqueName: \"kubernetes.io/projected/41a3a20b-ba1a-4e98-838e-0c1f728289b9-kube-api-access-wvg6b\") pod \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") "
Jul 08 23:35:10.879078 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.878983    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-model-cache\") pod \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") "
Jul 08 23:35:10.879078 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.879003    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tls-certs\") pod \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") "
Jul 08 23:35:10.879244 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.879221    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-model-cache" (OuterVolumeSpecName: "model-cache") pod "41a3a20b-ba1a-4e98-838e-0c1f728289b9" (UID: "41a3a20b-ba1a-4e98-838e-0c1f728289b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:35:10.879401 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.879380    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-model-cache\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:10.881232 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.881198    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "41a3a20b-ba1a-4e98-838e-0c1f728289b9" (UID: "41a3a20b-ba1a-4e98-838e-0c1f728289b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:35:10.881409 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.881392    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-dshm" (OuterVolumeSpecName: "dshm") pod "41a3a20b-ba1a-4e98-838e-0c1f728289b9" (UID: "41a3a20b-ba1a-4e98-838e-0c1f728289b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:35:10.881479 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.881426    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a3a20b-ba1a-4e98-838e-0c1f728289b9-kube-api-access-wvg6b" (OuterVolumeSpecName: "kube-api-access-wvg6b") pod "41a3a20b-ba1a-4e98-838e-0c1f728289b9" (UID: "41a3a20b-ba1a-4e98-838e-0c1f728289b9"). InnerVolumeSpecName "kube-api-access-wvg6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:35:10.980543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.980520    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tmp-dir\") pod \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") "
Jul 08 23:35:10.980652 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.980591    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-home\") pod \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\" (UID: \"41a3a20b-ba1a-4e98-838e-0c1f728289b9\") "
Jul 08 23:35:10.980765 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.980748    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-dshm\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:10.980765 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.980762    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvg6b\" (UniqueName: \"kubernetes.io/projected/41a3a20b-ba1a-4e98-838e-0c1f728289b9-kube-api-access-wvg6b\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:10.980896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.980772    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tls-certs\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:10.980896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.980774    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "41a3a20b-ba1a-4e98-838e-0c1f728289b9" (UID: "41a3a20b-ba1a-4e98-838e-0c1f728289b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:35:10.980896 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:10.980786    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-home" (OuterVolumeSpecName: "home") pod "41a3a20b-ba1a-4e98-838e-0c1f728289b9" (UID: "41a3a20b-ba1a-4e98-838e-0c1f728289b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:35:11.081564 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.081541    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-tmp-dir\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:11.081564 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.081565    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41a3a20b-ba1a-4e98-838e-0c1f728289b9-home\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:11.448483 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.448417    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q" podUID="67ceed24-8045-4fa9-abd1-2f840a335662" containerName="main" containerID="cri-o://3ae0674b6928327597803fef1e25279eddc559667bf078ae8006427dd0b62c28" gracePeriod=45
Jul 08 23:35:11.696216 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.696186    2560 generic.go:356] "Generic (PLEG): container finished" podID="41a3a20b-ba1a-4e98-838e-0c1f728289b9" containerID="d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7" exitCode=0
Jul 08 23:35:11.696592 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.696226    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"
Jul 08 23:35:11.696592 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.696267    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz" event={"ID":"41a3a20b-ba1a-4e98-838e-0c1f728289b9","Type":"ContainerDied","Data":"d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7"}
Jul 08 23:35:11.696592 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.696315    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz" event={"ID":"41a3a20b-ba1a-4e98-838e-0c1f728289b9","Type":"ContainerDied","Data":"b1460e92d3a76f243f214df85c80af3234e7eb6da67783b2130c713d1d9658a2"}
Jul 08 23:35:11.696592 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.696339    2560 scope.go:117] "RemoveContainer" containerID="d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7"
Jul 08 23:35:11.697826 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.697800    2560 generic.go:356] "Generic (PLEG): container finished" podID="67ceed24-8045-4fa9-abd1-2f840a335662" containerID="3ae0674b6928327597803fef1e25279eddc559667bf078ae8006427dd0b62c28" exitCode=0
Jul 08 23:35:11.697929 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.697870    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q" event={"ID":"67ceed24-8045-4fa9-abd1-2f840a335662","Type":"ContainerDied","Data":"3ae0674b6928327597803fef1e25279eddc559667bf078ae8006427dd0b62c28"}
Jul 08 23:35:11.697929 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.697900    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q" event={"ID":"67ceed24-8045-4fa9-abd1-2f840a335662","Type":"ContainerDied","Data":"e0db32920139f119d03093a421a7d2206767b9d07dc60ddd926648bc0358fdec"}
Jul 08 23:35:11.697929 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.697917    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0db32920139f119d03093a421a7d2206767b9d07dc60ddd926648bc0358fdec"
Jul 08 23:35:11.698172 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.698148    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:35:11.705350 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.705304    2560 scope.go:117] "RemoveContainer" containerID="d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7"
Jul 08 23:35:11.705643 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:35:11.705606    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7\": container with ID starting with d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7 not found: ID does not exist" containerID="d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7"
Jul 08 23:35:11.705725 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.705644    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7"} err="failed to get container status \"d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7\": rpc error: code = NotFound desc = could not find container \"d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7\": container with ID starting with d383344230c5236e5d698fa7d1e8bda6519295aaa6f4e76304496e3bdcdda5e7 not found: ID does not exist"
Jul 08 23:35:11.740427 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.736601    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"]
Jul 08 23:35:11.742948 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.742923    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-96223d/test-model-96223d-kserve-65794d6549-xc5qz"]
Jul 08 23:35:11.788239 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.788214    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-tmp-dir\") pod \"67ceed24-8045-4fa9-abd1-2f840a335662\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") "
Jul 08 23:35:11.788342 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.788264    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67ceed24-8045-4fa9-abd1-2f840a335662-tls-certs\") pod \"67ceed24-8045-4fa9-abd1-2f840a335662\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") "
Jul 08 23:35:11.788342 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.788283    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-model-cache\") pod \"67ceed24-8045-4fa9-abd1-2f840a335662\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") "
Jul 08 23:35:11.788342 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.788340    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-home\") pod \"67ceed24-8045-4fa9-abd1-2f840a335662\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") "
Jul 08 23:35:11.788542 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.788359    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-dshm\") pod \"67ceed24-8045-4fa9-abd1-2f840a335662\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") "
Jul 08 23:35:11.788542 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.788401    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbvsb\" (UniqueName: \"kubernetes.io/projected/67ceed24-8045-4fa9-abd1-2f840a335662-kube-api-access-bbvsb\") pod \"67ceed24-8045-4fa9-abd1-2f840a335662\" (UID: \"67ceed24-8045-4fa9-abd1-2f840a335662\") "
Jul 08 23:35:11.788651 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.788533    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "67ceed24-8045-4fa9-abd1-2f840a335662" (UID: "67ceed24-8045-4fa9-abd1-2f840a335662"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:35:11.788651 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.788618    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-home" (OuterVolumeSpecName: "home") pod "67ceed24-8045-4fa9-abd1-2f840a335662" (UID: "67ceed24-8045-4fa9-abd1-2f840a335662"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:35:11.788651 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.788631    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-model-cache" (OuterVolumeSpecName: "model-cache") pod "67ceed24-8045-4fa9-abd1-2f840a335662" (UID: "67ceed24-8045-4fa9-abd1-2f840a335662"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:35:11.790493 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.790445    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67ceed24-8045-4fa9-abd1-2f840a335662-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "67ceed24-8045-4fa9-abd1-2f840a335662" (UID: "67ceed24-8045-4fa9-abd1-2f840a335662"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:35:11.790576 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.790492    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67ceed24-8045-4fa9-abd1-2f840a335662-kube-api-access-bbvsb" (OuterVolumeSpecName: "kube-api-access-bbvsb") pod "67ceed24-8045-4fa9-abd1-2f840a335662" (UID: "67ceed24-8045-4fa9-abd1-2f840a335662"). InnerVolumeSpecName "kube-api-access-bbvsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:35:11.790576 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.790498    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-dshm" (OuterVolumeSpecName: "dshm") pod "67ceed24-8045-4fa9-abd1-2f840a335662" (UID: "67ceed24-8045-4fa9-abd1-2f840a335662"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:35:11.889939 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.889917    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67ceed24-8045-4fa9-abd1-2f840a335662-tls-certs\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:11.889939 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.889939    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-model-cache\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:11.890063 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.889947    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-home\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:11.890063 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.889954    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-dshm\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:11.890063 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.889962    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbvsb\" (UniqueName: \"kubernetes.io/projected/67ceed24-8045-4fa9-abd1-2f840a335662-kube-api-access-bbvsb\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:11.890063 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:11.889971    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67ceed24-8045-4fa9-abd1-2f840a335662-tmp-dir\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:12.704128 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:12.704101    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"
Jul 08 23:35:12.724055 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:12.724029    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"]
Jul 08 23:35:12.729928 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:12.729907    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-e9d26a/test-model-e9d26a-kserve-ddb9c499f-cgb9q"]
Jul 08 23:35:13.704408 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:13.704374    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41a3a20b-ba1a-4e98-838e-0c1f728289b9" path="/var/lib/kubelet/pods/41a3a20b-ba1a-4e98-838e-0c1f728289b9/volumes"
Jul 08 23:35:13.704792 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:13.704776    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67ceed24-8045-4fa9-abd1-2f840a335662" path="/var/lib/kubelet/pods/67ceed24-8045-4fa9-abd1-2f840a335662/volumes"
Jul 08 23:35:38.697563 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:38.697529    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:39.163989 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:39.163904    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:39.351084 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:39.351050    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:39.753963 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:39.753932    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:39.874707 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:39.874672    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:39.979688 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:39.979655    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:40.194978 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.194943    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"]
Jul 08 23:35:40.195212 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.195186    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg" podUID="4d103efd-2ed2-4c89-b723-c7a07c0a5ee2" containerName="maas-api" containerID="cri-o://5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2" gracePeriod=30
Jul 08 23:35:40.227113 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.227079    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:40.445448 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.445385    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:35:40.532631 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.532603    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-maas-api-tls\") pod \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") "
Jul 08 23:35:40.532804 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.532663    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrf9p\" (UniqueName: \"kubernetes.io/projected/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-kube-api-access-hrf9p\") pod \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") "
Jul 08 23:35:40.532804 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.532703    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-openshift-service-ca\") pod \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\" (UID: \"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2\") "
Jul 08 23:35:40.533053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.533027    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4d103efd-2ed2-4c89-b723-c7a07c0a5ee2" (UID: "4d103efd-2ed2-4c89-b723-c7a07c0a5ee2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:35:40.534733 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.534708    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4d103efd-2ed2-4c89-b723-c7a07c0a5ee2" (UID: "4d103efd-2ed2-4c89-b723-c7a07c0a5ee2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:35:40.534847 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.534749    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-kube-api-access-hrf9p" (OuterVolumeSpecName: "kube-api-access-hrf9p") pod "4d103efd-2ed2-4c89-b723-c7a07c0a5ee2" (UID: "4d103efd-2ed2-4c89-b723-c7a07c0a5ee2"). InnerVolumeSpecName "kube-api-access-hrf9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:35:40.634080 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.634045    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-maas-api-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:40.634080 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.634073    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrf9p\" (UniqueName: \"kubernetes.io/projected/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-kube-api-access-hrf9p\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:40.634080 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.634082    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2-openshift-service-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:35:40.839670 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.839569    2560 generic.go:356] "Generic (PLEG): container finished" podID="4d103efd-2ed2-4c89-b723-c7a07c0a5ee2" containerID="5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2" exitCode=0
Jul 08 23:35:40.839670 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.839613    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"
Jul 08 23:35:40.839670 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.839634    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg" event={"ID":"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2","Type":"ContainerDied","Data":"5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2"}
Jul 08 23:35:40.840138 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.839689    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg" event={"ID":"4d103efd-2ed2-4c89-b723-c7a07c0a5ee2","Type":"ContainerDied","Data":"f1d5b58b63486af0b3b268e79096c950ea5dd392bba0e958c6b2ae8538e38fc0"}
Jul 08 23:35:40.840138 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.839711    2560 scope.go:117] "RemoveContainer" containerID="5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2"
Jul 08 23:35:40.850614 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.850594    2560 scope.go:117] "RemoveContainer" containerID="5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2"
Jul 08 23:35:40.850855 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:35:40.850838    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2\": container with ID starting with 5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2 not found: ID does not exist" containerID="5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2"
Jul 08 23:35:40.850911 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.850863    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2"} err="failed to get container status \"5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2\": rpc error: code = NotFound desc = could not find container \"5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2\": container with ID starting with 5643f1810ac336a820c1f1700849dd6369d801ca27577fc25c5f62590154f5b2 not found: ID does not exist"
Jul 08 23:35:40.861915 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.861890    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"]
Jul 08 23:35:40.865587 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:40.865568    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e9d26a-7d7dd5f558-xcsgg"]
Jul 08 23:35:41.704526 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:41.704493    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d103efd-2ed2-4c89-b723-c7a07c0a5ee2" path="/var/lib/kubelet/pods/4d103efd-2ed2-4c89-b723-c7a07c0a5ee2/volumes"
Jul 08 23:35:45.818913 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:45.818875    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:45.884242 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:45.884216    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:46.075137 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:46.075032    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:46.631400 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:46.631366    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:46.918778 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:46.918697    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:47.023106 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:47.023065    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:49.718246 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.718217    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:49.787082 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787043    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"]
Jul 08 23:35:49.787574 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787550    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9bfca4b-d0ec-4ffb-8100-729e3198964f" containerName="maas-api"
Jul 08 23:35:49.787574 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787568    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9bfca4b-d0ec-4ffb-8100-729e3198964f" containerName="maas-api"
Jul 08 23:35:49.787738 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787585    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67ceed24-8045-4fa9-abd1-2f840a335662" containerName="main"
Jul 08 23:35:49.787738 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787592    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="67ceed24-8045-4fa9-abd1-2f840a335662" containerName="main"
Jul 08 23:35:49.787738 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787613    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d103efd-2ed2-4c89-b723-c7a07c0a5ee2" containerName="maas-api"
Jul 08 23:35:49.787738 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787619    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d103efd-2ed2-4c89-b723-c7a07c0a5ee2" containerName="maas-api"
Jul 08 23:35:49.787738 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787638    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a3a20b-ba1a-4e98-838e-0c1f728289b9" containerName="main"
Jul 08 23:35:49.787738 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787643    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a3a20b-ba1a-4e98-838e-0c1f728289b9" containerName="main"
Jul 08 23:35:49.787738 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787699    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="67ceed24-8045-4fa9-abd1-2f840a335662" containerName="main"
Jul 08 23:35:49.787738 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787711    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a3a20b-ba1a-4e98-838e-0c1f728289b9" containerName="main"
Jul 08 23:35:49.787738 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787718    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9bfca4b-d0ec-4ffb-8100-729e3198964f" containerName="maas-api"
Jul 08 23:35:49.787738 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.787725    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d103efd-2ed2-4c89-b723-c7a07c0a5ee2" containerName="maas-api"
Jul 08 23:35:49.792252 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.792234    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:49.794493 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.794462    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-0a9b6b-serving-cert\""
Jul 08 23:35:49.814026 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.813994    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:49.814315 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.814266    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"]
Jul 08 23:35:49.868543 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.868496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:49.914506 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.914460    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f9cm\" (UniqueName: \"kubernetes.io/projected/25cd88bb-31d4-4e0e-a124-205b37239a9a-kube-api-access-4f9cm\") pod \"maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:49.914616 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.914521    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25cd88bb-31d4-4e0e-a124-205b37239a9a-maas-api-tls\") pod \"maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:49.914616 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:49.914555    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cd88bb-31d4-4e0e-a124-205b37239a9a-openshift-service-ca\") pod \"maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:50.015436 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.015372    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f9cm\" (UniqueName: \"kubernetes.io/projected/25cd88bb-31d4-4e0e-a124-205b37239a9a-kube-api-access-4f9cm\") pod \"maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:50.015436 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.015406    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25cd88bb-31d4-4e0e-a124-205b37239a9a-maas-api-tls\") pod \"maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:50.015436 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.015431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cd88bb-31d4-4e0e-a124-205b37239a9a-openshift-service-ca\") pod \"maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:50.016077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.016057    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cd88bb-31d4-4e0e-a124-205b37239a9a-openshift-service-ca\") pod \"maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:50.017820 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.017800    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25cd88bb-31d4-4e0e-a124-205b37239a9a-maas-api-tls\") pod \"maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:50.020088 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.020058    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:50.023683 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.023662    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f9cm\" (UniqueName: \"kubernetes.io/projected/25cd88bb-31d4-4e0e-a124-205b37239a9a-kube-api-access-4f9cm\") pod \"maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:50.104313 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.104283    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:50.237834 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.237810    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"]
Jul 08 23:35:50.239944 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:35:50.239917    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25cd88bb_31d4_4e0e_a124_205b37239a9a.slice/crio-301bcc4db1f7497b1a91656a56e8d25f0601e42445141bbacfcc57481ec83605 WatchSource:0}: Error finding container 301bcc4db1f7497b1a91656a56e8d25f0601e42445141bbacfcc57481ec83605: Status 404 returned error can't find the container with id 301bcc4db1f7497b1a91656a56e8d25f0601e42445141bbacfcc57481ec83605
Jul 08 23:35:50.279046 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.278980    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:50.380734 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.380696    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:50.888631 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.888597    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt" event={"ID":"25cd88bb-31d4-4e0e-a124-205b37239a9a","Type":"ContainerStarted","Data":"38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c"}
Jul 08 23:35:50.888932 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.888650    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt" event={"ID":"25cd88bb-31d4-4e0e-a124-205b37239a9a","Type":"ContainerStarted","Data":"301bcc4db1f7497b1a91656a56e8d25f0601e42445141bbacfcc57481ec83605"}
Jul 08 23:35:50.888932 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.888699    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:50.903879 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:50.903816    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt" podStartSLOduration=1.3235775539999999 podStartE2EDuration="1.903798662s" totalImagesPullingTime="580.221108ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:35:50.903761724 +0000 UTC m=+1969.833795700" watchObservedRunningTime="2026-07-08 23:35:50.903798662 +0000 UTC m=+1969.833832634"
Jul 08 23:35:54.198795 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:54.198718    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:54.644283 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:54.644199    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:54.921245 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:54.921215    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:55.082398 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:55.082369    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:56.898772 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:56.898745    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:35:57.765280 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:57.765241    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:57.868820 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:57.868780    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:57.998215 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:57.998186    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"]
Jul 08 23:35:58.002286 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.002262    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.004885 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.004864    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-615023-serving-cert\""
Jul 08 23:35:58.013841 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.013819    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"]
Jul 08 23:35:58.040971 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.040919    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:58.145121 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.145087    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:58.185970 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.185946    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0748cda9-5536-47ea-9eef-8d3502345761-maas-api-tls\") pod \"maas-api-e2e-api-b-615023-6b5c59767f-lz66v\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.186109 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.185998    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wrlm\" (UniqueName: \"kubernetes.io/projected/0748cda9-5536-47ea-9eef-8d3502345761-kube-api-access-6wrlm\") pod \"maas-api-e2e-api-b-615023-6b5c59767f-lz66v\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.186109 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.186081    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0748cda9-5536-47ea-9eef-8d3502345761-openshift-service-ca\") pod \"maas-api-e2e-api-b-615023-6b5c59767f-lz66v\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.286738 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.286711    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0748cda9-5536-47ea-9eef-8d3502345761-maas-api-tls\") pod \"maas-api-e2e-api-b-615023-6b5c59767f-lz66v\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.286855 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.286758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wrlm\" (UniqueName: \"kubernetes.io/projected/0748cda9-5536-47ea-9eef-8d3502345761-kube-api-access-6wrlm\") pod \"maas-api-e2e-api-b-615023-6b5c59767f-lz66v\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.286855 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.286792    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0748cda9-5536-47ea-9eef-8d3502345761-openshift-service-ca\") pod \"maas-api-e2e-api-b-615023-6b5c59767f-lz66v\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.287415 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.287388    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0748cda9-5536-47ea-9eef-8d3502345761-openshift-service-ca\") pod \"maas-api-e2e-api-b-615023-6b5c59767f-lz66v\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.289109 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.289091    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0748cda9-5536-47ea-9eef-8d3502345761-maas-api-tls\") pod \"maas-api-e2e-api-b-615023-6b5c59767f-lz66v\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.297884 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.297829    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wrlm\" (UniqueName: \"kubernetes.io/projected/0748cda9-5536-47ea-9eef-8d3502345761-kube-api-access-6wrlm\") pod \"maas-api-e2e-api-b-615023-6b5c59767f-lz66v\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.314774 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.314754    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.446057 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.446034    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"]
Jul 08 23:35:58.448424 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:35:58.448400    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0748cda9_5536_47ea_9eef_8d3502345761.slice/crio-89097d3d87378bae30c859f389340c38c3eb8d22721fbcd521df24e1f0e9c302 WatchSource:0}: Error finding container 89097d3d87378bae30c859f389340c38c3eb8d22721fbcd521df24e1f0e9c302: Status 404 returned error can't find the container with id 89097d3d87378bae30c859f389340c38c3eb8d22721fbcd521df24e1f0e9c302
Jul 08 23:35:58.515172 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.515145    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:58.615615 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.615529    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:35:58.937285 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.937250    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v" event={"ID":"0748cda9-5536-47ea-9eef-8d3502345761","Type":"ContainerStarted","Data":"f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb"}
Jul 08 23:35:58.937454 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.937310    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v" event={"ID":"0748cda9-5536-47ea-9eef-8d3502345761","Type":"ContainerStarted","Data":"89097d3d87378bae30c859f389340c38c3eb8d22721fbcd521df24e1f0e9c302"}
Jul 08 23:35:58.937454 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.937379    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:35:58.956725 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:35:58.956685    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v" podStartSLOduration=1.6233763890000001 podStartE2EDuration="1.956670311s" totalImagesPullingTime="333.293922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:35:58.954887317 +0000 UTC m=+1977.884921290" watchObservedRunningTime="2026-07-08 23:35:58.956670311 +0000 UTC m=+1977.886704284"
Jul 08 23:36:02.237131 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:02.237098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:04.948115 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:04.948080    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:36:10.164462 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:10.164426    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:10.601005 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:10.600967    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:10.760994 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:10.760957    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:10.863252 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:10.863158    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:10.991172 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:10.991140    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"]
Jul 08 23:36:10.991422 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:10.991386    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt" podUID="25cd88bb-31d4-4e0e-a124-205b37239a9a" containerName="maas-api" containerID="cri-o://38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c" gracePeriod=30
Jul 08 23:36:11.016256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:11.016227    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:11.241924 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:11.241902    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:36:11.296388 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:11.296354    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f9cm\" (UniqueName: \"kubernetes.io/projected/25cd88bb-31d4-4e0e-a124-205b37239a9a-kube-api-access-4f9cm\") pod \"25cd88bb-31d4-4e0e-a124-205b37239a9a\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") "
Jul 08 23:36:11.296572 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:11.296452    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cd88bb-31d4-4e0e-a124-205b37239a9a-openshift-service-ca\") pod \"25cd88bb-31d4-4e0e-a124-205b37239a9a\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") "
Jul 08 23:36:11.296572 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:11.296538    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25cd88bb-31d4-4e0e-a124-205b37239a9a-maas-api-tls\") pod \"25cd88bb-31d4-4e0e-a124-205b37239a9a\" (UID: \"25cd88bb-31d4-4e0e-a124-205b37239a9a\") "
Jul 08 23:36:11.296884 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:11.296859    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25cd88bb-31d4-4e0e-a124-205b37239a9a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "25cd88bb-31d4-4e0e-a124-205b37239a9a" (UID: "25cd88bb-31d4-4e0e-a124-205b37239a9a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:36:11.298565 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:11.298537    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25cd88bb-31d4-4e0e-a124-205b37239a9a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "25cd88bb-31d4-4e0e-a124-205b37239a9a" (UID: "25cd88bb-31d4-4e0e-a124-205b37239a9a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:36:11.298693 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:11.298592    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25cd88bb-31d4-4e0e-a124-205b37239a9a-kube-api-access-4f9cm" (OuterVolumeSpecName: "kube-api-access-4f9cm") pod "25cd88bb-31d4-4e0e-a124-205b37239a9a" (UID: "25cd88bb-31d4-4e0e-a124-205b37239a9a"). InnerVolumeSpecName "kube-api-access-4f9cm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:36:11.397800 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:11.397767    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25cd88bb-31d4-4e0e-a124-205b37239a9a-openshift-service-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:36:11.397800 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:11.397794    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25cd88bb-31d4-4e0e-a124-205b37239a9a-maas-api-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:36:11.397800 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:11.397803    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f9cm\" (UniqueName: \"kubernetes.io/projected/25cd88bb-31d4-4e0e-a124-205b37239a9a-kube-api-access-4f9cm\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:36:12.011952 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:12.011860    2560 generic.go:356] "Generic (PLEG): container finished" podID="25cd88bb-31d4-4e0e-a124-205b37239a9a" containerID="38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c" exitCode=0
Jul 08 23:36:12.011952 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:12.011926    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt" event={"ID":"25cd88bb-31d4-4e0e-a124-205b37239a9a","Type":"ContainerDied","Data":"38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c"}
Jul 08 23:36:12.011952 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:12.011931    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"
Jul 08 23:36:12.012242 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:12.011978    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt" event={"ID":"25cd88bb-31d4-4e0e-a124-205b37239a9a","Type":"ContainerDied","Data":"301bcc4db1f7497b1a91656a56e8d25f0601e42445141bbacfcc57481ec83605"}
Jul 08 23:36:12.012242 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:12.012008    2560 scope.go:117] "RemoveContainer" containerID="38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c"
Jul 08 23:36:12.022037 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:12.022016    2560 scope.go:117] "RemoveContainer" containerID="38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c"
Jul 08 23:36:12.022304 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:36:12.022283    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c\": container with ID starting with 38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c not found: ID does not exist" containerID="38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c"
Jul 08 23:36:12.022382 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:12.022316    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c"} err="failed to get container status \"38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c\": rpc error: code = NotFound desc = could not find container \"38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c\": container with ID starting with 38bc3b78ba0c7d2736870599fa5bd1203a11ee62f73f22350a1d1cb1d826099c not found: ID does not exist"
Jul 08 23:36:12.032435 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:12.032409    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"]
Jul 08 23:36:12.036651 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:12.036627    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a9b6b-6d4646cff-w88lt"]
Jul 08 23:36:13.704713 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:13.704683    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25cd88bb-31d4-4e0e-a124-205b37239a9a" path="/var/lib/kubelet/pods/25cd88bb-31d4-4e0e-a124-205b37239a9a/volumes"
Jul 08 23:36:16.226364 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:16.226324    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:16.326818 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:16.326789    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:16.478722 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:16.478643    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:16.859020 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:16.858938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:17.222872 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.222833    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:17.323814 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.323776    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:17.423952 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.423911    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:17.516508 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.516399    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"]
Jul 08 23:36:17.516685 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.516657    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v" podUID="0748cda9-5536-47ea-9eef-8d3502345761" containerName="maas-api" containerID="cri-o://f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb" gracePeriod=30
Jul 08 23:36:17.540081 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.540045    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:17.759390 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.759362    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:36:17.850600 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.850521    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wrlm\" (UniqueName: \"kubernetes.io/projected/0748cda9-5536-47ea-9eef-8d3502345761-kube-api-access-6wrlm\") pod \"0748cda9-5536-47ea-9eef-8d3502345761\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") "
Jul 08 23:36:17.850736 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.850625    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0748cda9-5536-47ea-9eef-8d3502345761-openshift-service-ca\") pod \"0748cda9-5536-47ea-9eef-8d3502345761\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") "
Jul 08 23:36:17.850736 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.850654    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0748cda9-5536-47ea-9eef-8d3502345761-maas-api-tls\") pod \"0748cda9-5536-47ea-9eef-8d3502345761\" (UID: \"0748cda9-5536-47ea-9eef-8d3502345761\") "
Jul 08 23:36:17.851015 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.850986    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0748cda9-5536-47ea-9eef-8d3502345761-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0748cda9-5536-47ea-9eef-8d3502345761" (UID: "0748cda9-5536-47ea-9eef-8d3502345761"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:36:17.852742 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.852718    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0748cda9-5536-47ea-9eef-8d3502345761-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0748cda9-5536-47ea-9eef-8d3502345761" (UID: "0748cda9-5536-47ea-9eef-8d3502345761"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:36:17.852842 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.852778    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0748cda9-5536-47ea-9eef-8d3502345761-kube-api-access-6wrlm" (OuterVolumeSpecName: "kube-api-access-6wrlm") pod "0748cda9-5536-47ea-9eef-8d3502345761" (UID: "0748cda9-5536-47ea-9eef-8d3502345761"). InnerVolumeSpecName "kube-api-access-6wrlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:36:17.952036 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.952002    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0748cda9-5536-47ea-9eef-8d3502345761-openshift-service-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:36:17.952036 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.952030    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0748cda9-5536-47ea-9eef-8d3502345761-maas-api-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:36:17.952036 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:17.952043    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wrlm\" (UniqueName: \"kubernetes.io/projected/0748cda9-5536-47ea-9eef-8d3502345761-kube-api-access-6wrlm\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:36:18.041511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:18.041458    2560 generic.go:356] "Generic (PLEG): container finished" podID="0748cda9-5536-47ea-9eef-8d3502345761" containerID="f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb" exitCode=0
Jul 08 23:36:18.041511 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:18.041489    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v" event={"ID":"0748cda9-5536-47ea-9eef-8d3502345761","Type":"ContainerDied","Data":"f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb"}
Jul 08 23:36:18.041730 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:18.041516    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"
Jul 08 23:36:18.041730 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:18.041541    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v" event={"ID":"0748cda9-5536-47ea-9eef-8d3502345761","Type":"ContainerDied","Data":"89097d3d87378bae30c859f389340c38c3eb8d22721fbcd521df24e1f0e9c302"}
Jul 08 23:36:18.041730 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:18.041563    2560 scope.go:117] "RemoveContainer" containerID="f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb"
Jul 08 23:36:18.050890 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:18.050875    2560 scope.go:117] "RemoveContainer" containerID="f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb"
Jul 08 23:36:18.051155 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:36:18.051136    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb\": container with ID starting with f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb not found: ID does not exist" containerID="f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb"
Jul 08 23:36:18.051222 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:18.051163    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb"} err="failed to get container status \"f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb\": rpc error: code = NotFound desc = could not find container \"f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb\": container with ID starting with f8eb0979c8e90d715deae6664c71a81b4322d4ef8adaf0d96a8d206f4bdd8dbb not found: ID does not exist"
Jul 08 23:36:18.061408 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:18.061280    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"]
Jul 08 23:36:18.063698 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:18.063677    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-615023-6b5c59767f-lz66v"]
Jul 08 23:36:19.705691 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:19.705652    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0748cda9-5536-47ea-9eef-8d3502345761" path="/var/lib/kubelet/pods/0748cda9-5536-47ea-9eef-8d3502345761/volumes"
Jul 08 23:36:22.957716 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:22.957663    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:23.057552 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:23.057514    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:23.213747 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:23.213660    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:23.738486 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:23.738434    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:24.034053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:24.033963    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:24.187451 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:24.187409    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:26.937439 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:26.937403    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:27.015814 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.015785    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"]
Jul 08 23:36:27.016252 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.016228    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0748cda9-5536-47ea-9eef-8d3502345761" containerName="maas-api"
Jul 08 23:36:27.016252 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.016253    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0748cda9-5536-47ea-9eef-8d3502345761" containerName="maas-api"
Jul 08 23:36:27.016388 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.016312    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25cd88bb-31d4-4e0e-a124-205b37239a9a" containerName="maas-api"
Jul 08 23:36:27.016388 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.016322    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="25cd88bb-31d4-4e0e-a124-205b37239a9a" containerName="maas-api"
Jul 08 23:36:27.016693 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.016407    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0748cda9-5536-47ea-9eef-8d3502345761" containerName="maas-api"
Jul 08 23:36:27.016693 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.016423    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="25cd88bb-31d4-4e0e-a124-205b37239a9a" containerName="maas-api"
Jul 08 23:36:27.018687 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.018660    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:27.020908 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.020891    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:27.023591 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.023566    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-45d83d-serving-cert\""
Jul 08 23:36:27.027677 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.027647    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"]
Jul 08 23:36:27.074759 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.074729    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:27.131828 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.131805    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fce968c-736e-40f3-a7af-d78711729045-maas-api-tls\") pod \"maas-api-e2e-shared-a-45d83d-996bd8774-89r2w\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:27.131943 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.131837    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fce968c-736e-40f3-a7af-d78711729045-openshift-service-ca\") pod \"maas-api-e2e-shared-a-45d83d-996bd8774-89r2w\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:27.131943 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.131857    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx7rf\" (UniqueName: \"kubernetes.io/projected/1fce968c-736e-40f3-a7af-d78711729045-kube-api-access-lx7rf\") pod \"maas-api-e2e-shared-a-45d83d-996bd8774-89r2w\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:27.233253 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.233192    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fce968c-736e-40f3-a7af-d78711729045-openshift-service-ca\") pod \"maas-api-e2e-shared-a-45d83d-996bd8774-89r2w\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:27.233253 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.233223    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx7rf\" (UniqueName: \"kubernetes.io/projected/1fce968c-736e-40f3-a7af-d78711729045-kube-api-access-lx7rf\") pod \"maas-api-e2e-shared-a-45d83d-996bd8774-89r2w\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:27.233394 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.233299    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fce968c-736e-40f3-a7af-d78711729045-maas-api-tls\") pod \"maas-api-e2e-shared-a-45d83d-996bd8774-89r2w\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:27.233394 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:36:27.233385    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-serving-cert: secret "maas-api-e2e-shared-a-45d83d-serving-cert" not found
Jul 08 23:36:27.233458 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:36:27.233434    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fce968c-736e-40f3-a7af-d78711729045-maas-api-tls podName:1fce968c-736e-40f3-a7af-d78711729045 nodeName:}" failed. No retries permitted until 2026-07-08 23:36:27.733417804 +0000 UTC m=+2006.663451756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1fce968c-736e-40f3-a7af-d78711729045-maas-api-tls") pod "maas-api-e2e-shared-a-45d83d-996bd8774-89r2w" (UID: "1fce968c-736e-40f3-a7af-d78711729045") : secret "maas-api-e2e-shared-a-45d83d-serving-cert" not found
Jul 08 23:36:27.233907 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.233886    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fce968c-736e-40f3-a7af-d78711729045-openshift-service-ca\") pod \"maas-api-e2e-shared-a-45d83d-996bd8774-89r2w\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:27.242023 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.241999    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx7rf\" (UniqueName: \"kubernetes.io/projected/1fce968c-736e-40f3-a7af-d78711729045-kube-api-access-lx7rf\") pod \"maas-api-e2e-shared-a-45d83d-996bd8774-89r2w\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:27.487519 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.487415    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:27.585347 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.585311    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:27.688492 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.688447    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:27.738077 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.738003    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fce968c-736e-40f3-a7af-d78711729045-maas-api-tls\") pod \"maas-api-e2e-shared-a-45d83d-996bd8774-89r2w\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:27.740426 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.740398    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fce968c-736e-40f3-a7af-d78711729045-maas-api-tls\") pod \"maas-api-e2e-shared-a-45d83d-996bd8774-89r2w\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:27.932702 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:27.932672    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:28.069503 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:28.065413    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"]
Jul 08 23:36:28.071219 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:36:28.071195    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fce968c_736e_40f3_a7af_d78711729045.slice/crio-dbce8e205c1f690e5f192587a6d4bec9cd77298493ad97c9f4a014f677f1e18c WatchSource:0}: Error finding container dbce8e205c1f690e5f192587a6d4bec9cd77298493ad97c9f4a014f677f1e18c: Status 404 returned error can't find the container with id dbce8e205c1f690e5f192587a6d4bec9cd77298493ad97c9f4a014f677f1e18c
Jul 08 23:36:28.095492 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:28.095445    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w" event={"ID":"1fce968c-736e-40f3-a7af-d78711729045","Type":"ContainerStarted","Data":"dbce8e205c1f690e5f192587a6d4bec9cd77298493ad97c9f4a014f677f1e18c"}
Jul 08 23:36:29.101424 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:29.101393    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w" event={"ID":"1fce968c-736e-40f3-a7af-d78711729045","Type":"ContainerStarted","Data":"fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612"}
Jul 08 23:36:29.101837 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:29.101512    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:29.119925 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:29.119884    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w" podStartSLOduration=2.662331243 podStartE2EDuration="3.119872334s" totalImagesPullingTime="457.541091ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:36:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:36:29.119000073 +0000 UTC m=+2008.049034047" watchObservedRunningTime="2026-07-08 23:36:29.119872334 +0000 UTC m=+2008.049906307"
Jul 08 23:36:31.422868 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:31.422821    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:31.743494 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:31.743389    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:32.019308 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:32.019227    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:32.456003 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:32.455964    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:34.982961 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:34.982922    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:35.051928 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.051891    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"]
Jul 08 23:36:35.055358 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.055335    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:35.058022 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.057999    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-59ce67-serving-cert\""
Jul 08 23:36:35.064978 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.064959    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"]
Jul 08 23:36:35.099249 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.099224    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-maas-api-tls\") pod \"maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:35.099349 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.099260    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:35.099349 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.099281    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4vqn\" (UniqueName: \"kubernetes.io/projected/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-kube-api-access-k4vqn\") pod \"maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:35.112051 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.112027    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:36:35.200398 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.200366    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-maas-api-tls\") pod \"maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:35.200550 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.200411    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:35.200550 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.200431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4vqn\" (UniqueName: \"kubernetes.io/projected/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-kube-api-access-k4vqn\") pod \"maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:35.201050 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.201027    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:35.202717 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.202700    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-maas-api-tls\") pod \"maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:35.209044 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.209022    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4vqn\" (UniqueName: \"kubernetes.io/projected/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-kube-api-access-k4vqn\") pod \"maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:35.367668 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.367597    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:35.500372 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:35.500345    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"]
Jul 08 23:36:35.502141 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:36:35.502113    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfae46a_8f9a_4445_94a8_70da6eaf5eb8.slice/crio-e84a98c2f2d73073226ad24ad96e33f872c656e204fc343d1d574f8fee1430ce WatchSource:0}: Error finding container e84a98c2f2d73073226ad24ad96e33f872c656e204fc343d1d574f8fee1430ce: Status 404 returned error can't find the container with id e84a98c2f2d73073226ad24ad96e33f872c656e204fc343d1d574f8fee1430ce
Jul 08 23:36:36.136103 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:36.136018    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl" event={"ID":"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8","Type":"ContainerStarted","Data":"f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907"}
Jul 08 23:36:36.136103 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:36.136073    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl" event={"ID":"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8","Type":"ContainerStarted","Data":"e84a98c2f2d73073226ad24ad96e33f872c656e204fc343d1d574f8fee1430ce"}
Jul 08 23:36:36.136627 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:36.136141    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:36.155650 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:36.155599    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl" podStartSLOduration=0.88987048 podStartE2EDuration="1.155586369s" totalImagesPullingTime="265.715889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:36:36.153233877 +0000 UTC m=+2015.083267852" watchObservedRunningTime="2026-07-08 23:36:36.155586369 +0000 UTC m=+2015.085620342"
Jul 08 23:36:36.171146 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:36.171116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:39.471049 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:39.471015    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:42.146814 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:42.146774    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:36:49.070940 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:49.070909    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:49.172771 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:49.172737    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:50.482240 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:50.482206    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:50.582525 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:50.582459    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:56.350510 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:56.350458    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:36:56.405781 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:36:56.405749    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:37:00.601918 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.601891    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"]
Jul 08 23:37:00.605710 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.605690    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.608371 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.608352    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"openshift-service-ca.crt\""
Jul 08 23:37:00.609286 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.609268    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"default-dockercfg-lblj9\""
Jul 08 23:37:00.609378 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.609302    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"kube-root-ca.crt\""
Jul 08 23:37:00.609378 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.609299    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-45d83d\"/\"auth-test-model-45d83d-kserve-self-signed-certs\""
Jul 08 23:37:00.615279 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.615260    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"]
Jul 08 23:37:00.630432 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.630404    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:37:00.737932 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.737897    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tmp-dir\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.737932 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.737935    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-home\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.738120 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.737963    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tls-certs\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.738120 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.738002    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-dshm\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.738120 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.738082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj596\" (UniqueName: \"kubernetes.io/projected/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-kube-api-access-gj596\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.738120 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.738110    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-model-cache\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.839362 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.839320    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tmp-dir\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.839362 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.839364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-home\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.839590 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.839412    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tls-certs\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.839590 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.839443    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-dshm\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.839590 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.839545    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj596\" (UniqueName: \"kubernetes.io/projected/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-kube-api-access-gj596\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.839590 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.839580    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-model-cache\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.839785 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.839747    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-home\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.839846 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.839799    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tmp-dir\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.839975 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.839951    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-model-cache\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.842010 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.841989    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-dshm\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.842098 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.842087    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tls-certs\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.846784 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.846763    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj596\" (UniqueName: \"kubernetes.io/projected/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-kube-api-access-gj596\") pod \"auth-test-model-45d83d-kserve-76fc8784d7-94fbc\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") " pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:00.916354 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:00.916291    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:01.044785 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:01.044758    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"]
Jul 08 23:37:01.046775 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:37:01.046740    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod452327dc_ef69_44a8_8e9d_ff0c7ba69c19.slice/crio-d0e5c339351e61a870d040542d52306bdaae4326ed2ddfc157f5c892a698d616 WatchSource:0}: Error finding container d0e5c339351e61a870d040542d52306bdaae4326ed2ddfc157f5c892a698d616: Status 404 returned error can't find the container with id d0e5c339351e61a870d040542d52306bdaae4326ed2ddfc157f5c892a698d616
Jul 08 23:37:01.262064 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:01.262026    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc" event={"ID":"452327dc-ef69-44a8-8e9d-ff0c7ba69c19","Type":"ContainerStarted","Data":"3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59"}
Jul 08 23:37:01.262064 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:01.262067    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc" event={"ID":"452327dc-ef69-44a8-8e9d-ff0c7ba69c19","Type":"ContainerStarted","Data":"d0e5c339351e61a870d040542d52306bdaae4326ed2ddfc157f5c892a698d616"}
Jul 08 23:37:01.279482 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:01.279421    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc" podStartSLOduration=1.279403801 podStartE2EDuration="1.279403801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:37:01.277616979 +0000 UTC m=+2040.207650953" watchObservedRunningTime="2026-07-08 23:37:01.279403801 +0000 UTC m=+2040.209437776"
Jul 08 23:37:10.916752 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:10.916717    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:10.916752 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:10.916755    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:10.929216 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:10.929191    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:11.322842 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:11.322817    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:37:12.195009 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:12.194981    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:37:18.396270 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.396234    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"]
Jul 08 23:37:18.400372 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.400353    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.403927 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.403865    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"openshift-service-ca.crt\""
Jul 08 23:37:18.404061 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.403884    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"kube-root-ca.crt\""
Jul 08 23:37:18.404061 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.403885    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"default-dockercfg-mw654\""
Jul 08 23:37:18.404189 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.403887    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"auth-test-model-59ce67-kserve-self-signed-certs\""
Jul 08 23:37:18.406336 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.406315    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"]
Jul 08 23:37:18.420835 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.420809    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:37:18.493100 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.493073    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tls-certs\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.493216 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.493111    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-dshm\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.493258 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.493219    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b525c\" (UniqueName: \"kubernetes.io/projected/af8e0c43-12b6-41b8-a198-8cb38b38bf68-kube-api-access-b525c\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.493258 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.493253    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tmp-dir\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.493331 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.493272    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-home\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.493331 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.493322    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-model-cache\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.594058 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.594020    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tmp-dir\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.594058 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.594058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-home\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.594294 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.594084    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-model-cache\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.594294 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.594122    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tls-certs\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.594294 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.594146    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-dshm\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.594294 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.594198    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b525c\" (UniqueName: \"kubernetes.io/projected/af8e0c43-12b6-41b8-a198-8cb38b38bf68-kube-api-access-b525c\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.594540 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.594459    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tmp-dir\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.594540 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.594526    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-model-cache\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.594645 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.594555    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-home\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.596301 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.596281    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-dshm\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.596648 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.596630    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tls-certs\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.601225 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.601205    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b525c\" (UniqueName: \"kubernetes.io/projected/af8e0c43-12b6-41b8-a198-8cb38b38bf68-kube-api-access-b525c\") pod \"auth-test-model-59ce67-kserve-69d7d46d59-5qldp\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") " pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.711280 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.711247    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:18.840282 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:18.840182    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"]
Jul 08 23:37:18.842484 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:37:18.842450    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf8e0c43_12b6_41b8_a198_8cb38b38bf68.slice/crio-b0599b2b8ca64c9b2c4714d06e464d0ffe3956c81a216393d752c9689b23c460 WatchSource:0}: Error finding container b0599b2b8ca64c9b2c4714d06e464d0ffe3956c81a216393d752c9689b23c460: Status 404 returned error can't find the container with id b0599b2b8ca64c9b2c4714d06e464d0ffe3956c81a216393d752c9689b23c460
Jul 08 23:37:19.349263 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:19.349222    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp" event={"ID":"af8e0c43-12b6-41b8-a198-8cb38b38bf68","Type":"ContainerStarted","Data":"70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222"}
Jul 08 23:37:19.349429 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:19.349279    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp" event={"ID":"af8e0c43-12b6-41b8-a198-8cb38b38bf68","Type":"ContainerStarted","Data":"b0599b2b8ca64c9b2c4714d06e464d0ffe3956c81a216393d752c9689b23c460"}
Jul 08 23:37:19.367356 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:19.367312    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp" podStartSLOduration=1.367296737 podStartE2EDuration="1.367296737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:37:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:37:19.365475676 +0000 UTC m=+2058.295509642" watchObservedRunningTime="2026-07-08 23:37:19.367296737 +0000 UTC m=+2058.297330711"
Jul 08 23:37:19.801030 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:19.800994    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:37:28.712355 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:28.712311    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:28.712772 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:28.712367    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:28.724812 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:28.724788    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:37:29.409323 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:37:29.409294    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:38:01.768869 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:38:01.768844    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:38:01.771209 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:38:01.771183    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:38:53.958022 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:38:53.957920    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:38:54.048121 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:38:54.048084    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:39:26.698757 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:26.698723    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"]
Jul 08 23:39:26.713708 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:26.713672    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:39:27.499915 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:27.499886    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"]
Jul 08 23:39:27.521953 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:27.521920    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:39:31.726134 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:31.726096    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:39:41.719050 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:41.719005    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc" podUID="452327dc-ef69-44a8-8e9d-ff0c7ba69c19" containerName="main" containerID="cri-o://3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59" gracePeriod=45
Jul 08 23:39:41.983060 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:41.982997    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:39:42.047182 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.047145    2560 generic.go:356] "Generic (PLEG): container finished" podID="452327dc-ef69-44a8-8e9d-ff0c7ba69c19" containerID="3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59" exitCode=0
Jul 08 23:39:42.047182 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.047181    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"
Jul 08 23:39:42.047402 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.047190    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc" event={"ID":"452327dc-ef69-44a8-8e9d-ff0c7ba69c19","Type":"ContainerDied","Data":"3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59"}
Jul 08 23:39:42.047402 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.047232    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc" event={"ID":"452327dc-ef69-44a8-8e9d-ff0c7ba69c19","Type":"ContainerDied","Data":"d0e5c339351e61a870d040542d52306bdaae4326ed2ddfc157f5c892a698d616"}
Jul 08 23:39:42.047402 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.047255    2560 scope.go:117] "RemoveContainer" containerID="3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59"
Jul 08 23:39:42.056142 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.056122    2560 scope.go:117] "RemoveContainer" containerID="3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59"
Jul 08 23:39:42.056390 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:39:42.056372    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59\": container with ID starting with 3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59 not found: ID does not exist" containerID="3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59"
Jul 08 23:39:42.056437 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.056402    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59"} err="failed to get container status \"3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59\": rpc error: code = NotFound desc = could not find container \"3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59\": container with ID starting with 3d5f58999921fe2264af9cacbf9009eef4e5ca2353d3fb026dc97f4934c70e59 not found: ID does not exist"
Jul 08 23:39:42.082391 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.082369    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tls-certs\") pod \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") "
Jul 08 23:39:42.082508 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.082407    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gj596\" (UniqueName: \"kubernetes.io/projected/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-kube-api-access-gj596\") pod \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") "
Jul 08 23:39:42.082508 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.082489    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-model-cache\") pod \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") "
Jul 08 23:39:42.082600 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.082539    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tmp-dir\") pod \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") "
Jul 08 23:39:42.082600 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.082573    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-home\") pod \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") "
Jul 08 23:39:42.082694 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.082604    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-dshm\") pod \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\" (UID: \"452327dc-ef69-44a8-8e9d-ff0c7ba69c19\") "
Jul 08 23:39:42.082797 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.082772    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-model-cache" (OuterVolumeSpecName: "model-cache") pod "452327dc-ef69-44a8-8e9d-ff0c7ba69c19" (UID: "452327dc-ef69-44a8-8e9d-ff0c7ba69c19"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:39:42.082917 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.082892    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-home" (OuterVolumeSpecName: "home") pod "452327dc-ef69-44a8-8e9d-ff0c7ba69c19" (UID: "452327dc-ef69-44a8-8e9d-ff0c7ba69c19"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:39:42.082975 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.082956    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "452327dc-ef69-44a8-8e9d-ff0c7ba69c19" (UID: "452327dc-ef69-44a8-8e9d-ff0c7ba69c19"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:39:42.083031 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.082979    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-home\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:42.083031 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.082997    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-model-cache\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:42.084796 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.084700    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "452327dc-ef69-44a8-8e9d-ff0c7ba69c19" (UID: "452327dc-ef69-44a8-8e9d-ff0c7ba69c19"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:39:42.084796 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.084762    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-kube-api-access-gj596" (OuterVolumeSpecName: "kube-api-access-gj596") pod "452327dc-ef69-44a8-8e9d-ff0c7ba69c19" (UID: "452327dc-ef69-44a8-8e9d-ff0c7ba69c19"). InnerVolumeSpecName "kube-api-access-gj596". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:39:42.085118 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.085100    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-dshm" (OuterVolumeSpecName: "dshm") pod "452327dc-ef69-44a8-8e9d-ff0c7ba69c19" (UID: "452327dc-ef69-44a8-8e9d-ff0c7ba69c19"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:39:42.184053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.184020    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tls-certs\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:42.184053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.184047    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj596\" (UniqueName: \"kubernetes.io/projected/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-kube-api-access-gj596\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:42.184053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.184056    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-tmp-dir\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:42.184272 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.184064    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/452327dc-ef69-44a8-8e9d-ff0c7ba69c19-dshm\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:42.370240 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.370212    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"]
Jul 08 23:39:42.373936 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.373911    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-45d83d/auth-test-model-45d83d-kserve-76fc8784d7-94fbc"]
Jul 08 23:39:42.522117 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.522072    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp" podUID="af8e0c43-12b6-41b8-a198-8cb38b38bf68" containerName="main" containerID="cri-o://70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222" gracePeriod=45
Jul 08 23:39:42.768240 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.768217    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:39:42.891233 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891145    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b525c\" (UniqueName: \"kubernetes.io/projected/af8e0c43-12b6-41b8-a198-8cb38b38bf68-kube-api-access-b525c\") pod \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") "
Jul 08 23:39:42.891233 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891192    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tls-certs\") pod \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") "
Jul 08 23:39:42.891233 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891224    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-model-cache\") pod \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") "
Jul 08 23:39:42.891554 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891264    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-home\") pod \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") "
Jul 08 23:39:42.891554 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891287    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-dshm\") pod \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") "
Jul 08 23:39:42.891554 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891312    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tmp-dir\") pod \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\" (UID: \"af8e0c43-12b6-41b8-a198-8cb38b38bf68\") "
Jul 08 23:39:42.891733 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891543    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-model-cache" (OuterVolumeSpecName: "model-cache") pod "af8e0c43-12b6-41b8-a198-8cb38b38bf68" (UID: "af8e0c43-12b6-41b8-a198-8cb38b38bf68"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:39:42.891733 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891566    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-home" (OuterVolumeSpecName: "home") pod "af8e0c43-12b6-41b8-a198-8cb38b38bf68" (UID: "af8e0c43-12b6-41b8-a198-8cb38b38bf68"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:39:42.891733 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891638    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "af8e0c43-12b6-41b8-a198-8cb38b38bf68" (UID: "af8e0c43-12b6-41b8-a198-8cb38b38bf68"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:39:42.891845 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891777    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-model-cache\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:42.891845 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891795    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-home\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:42.891845 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.891805    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tmp-dir\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:42.893542 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.893515    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-dshm" (OuterVolumeSpecName: "dshm") pod "af8e0c43-12b6-41b8-a198-8cb38b38bf68" (UID: "af8e0c43-12b6-41b8-a198-8cb38b38bf68"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:39:42.893758 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.893736    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "af8e0c43-12b6-41b8-a198-8cb38b38bf68" (UID: "af8e0c43-12b6-41b8-a198-8cb38b38bf68"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:39:42.893833 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.893799    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af8e0c43-12b6-41b8-a198-8cb38b38bf68-kube-api-access-b525c" (OuterVolumeSpecName: "kube-api-access-b525c") pod "af8e0c43-12b6-41b8-a198-8cb38b38bf68" (UID: "af8e0c43-12b6-41b8-a198-8cb38b38bf68"). InnerVolumeSpecName "kube-api-access-b525c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:39:42.992245 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.992216    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af8e0c43-12b6-41b8-a198-8cb38b38bf68-dshm\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:42.992245 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.992241    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b525c\" (UniqueName: \"kubernetes.io/projected/af8e0c43-12b6-41b8-a198-8cb38b38bf68-kube-api-access-b525c\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:42.992417 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:42.992255    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af8e0c43-12b6-41b8-a198-8cb38b38bf68-tls-certs\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:39:43.053028 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:43.052997    2560 generic.go:356] "Generic (PLEG): container finished" podID="af8e0c43-12b6-41b8-a198-8cb38b38bf68" containerID="70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222" exitCode=0
Jul 08 23:39:43.053176 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:43.053039    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"
Jul 08 23:39:43.053176 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:43.053079    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp" event={"ID":"af8e0c43-12b6-41b8-a198-8cb38b38bf68","Type":"ContainerDied","Data":"70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222"}
Jul 08 23:39:43.053176 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:43.053117    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp" event={"ID":"af8e0c43-12b6-41b8-a198-8cb38b38bf68","Type":"ContainerDied","Data":"b0599b2b8ca64c9b2c4714d06e464d0ffe3956c81a216393d752c9689b23c460"}
Jul 08 23:39:43.053176 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:43.053133    2560 scope.go:117] "RemoveContainer" containerID="70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222"
Jul 08 23:39:43.062531 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:43.062460    2560 scope.go:117] "RemoveContainer" containerID="70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222"
Jul 08 23:39:43.062731 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:39:43.062715    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222\": container with ID starting with 70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222 not found: ID does not exist" containerID="70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222"
Jul 08 23:39:43.062777 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:43.062740    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222"} err="failed to get container status \"70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222\": rpc error: code = NotFound desc = could not find container \"70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222\": container with ID starting with 70277fb641e791b6b1c7307e44574d1c2fc519c1652feb656b13d379d09fe222 not found: ID does not exist"
Jul 08 23:39:43.075203 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:43.075184    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"]
Jul 08 23:39:43.079480 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:43.079441    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/auth-test-model-59ce67-kserve-69d7d46d59-5qldp"]
Jul 08 23:39:43.704608 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:43.704577    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="452327dc-ef69-44a8-8e9d-ff0c7ba69c19" path="/var/lib/kubelet/pods/452327dc-ef69-44a8-8e9d-ff0c7ba69c19/volumes"
Jul 08 23:39:43.704972 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:39:43.704960    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af8e0c43-12b6-41b8-a198-8cb38b38bf68" path="/var/lib/kubelet/pods/af8e0c43-12b6-41b8-a198-8cb38b38bf68/volumes"
Jul 08 23:40:05.394424 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.394393    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"]
Jul 08 23:40:05.394939 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.394782    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af8e0c43-12b6-41b8-a198-8cb38b38bf68" containerName="main"
Jul 08 23:40:05.394939 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.394793    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="af8e0c43-12b6-41b8-a198-8cb38b38bf68" containerName="main"
Jul 08 23:40:05.394939 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.394804    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="452327dc-ef69-44a8-8e9d-ff0c7ba69c19" containerName="main"
Jul 08 23:40:05.394939 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.394809    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="452327dc-ef69-44a8-8e9d-ff0c7ba69c19" containerName="main"
Jul 08 23:40:05.394939 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.394874    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="452327dc-ef69-44a8-8e9d-ff0c7ba69c19" containerName="main"
Jul 08 23:40:05.394939 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.394882    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="af8e0c43-12b6-41b8-a198-8cb38b38bf68" containerName="main"
Jul 08 23:40:05.399609 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.399589    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.403051 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.403025    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"kube-root-ca.crt\""
Jul 08 23:40:05.403175 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.403049    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"default-dockercfg-mw654\""
Jul 08 23:40:05.403175 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.403062    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"openshift-service-ca.crt\""
Jul 08 23:40:05.403175 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.403025    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-59ce67\"/\"sub-test-model-59ce67-kserve-self-signed-certs\""
Jul 08 23:40:05.408631 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.408607    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"]
Jul 08 23:40:05.422999 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.422976    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:40:05.494339 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.494314    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-home\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.494448 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.494360    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b7337ed-35cd-42ff-85be-e6516693a4e6-tls-certs\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.494448 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.494382    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-tmp-dir\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.494576 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.494440    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-dshm\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.494576 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.494510    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krskk\" (UniqueName: \"kubernetes.io/projected/0b7337ed-35cd-42ff-85be-e6516693a4e6-kube-api-access-krskk\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.494646 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.494574    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-model-cache\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.595655 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.595613    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b7337ed-35cd-42ff-85be-e6516693a4e6-tls-certs\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.595823 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.595665    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-tmp-dir\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.595823 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.595694    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-dshm\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.595823 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.595745    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krskk\" (UniqueName: \"kubernetes.io/projected/0b7337ed-35cd-42ff-85be-e6516693a4e6-kube-api-access-krskk\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.595823 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.595782    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-model-cache\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.596037 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.595853    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-home\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.596149 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.596128    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-tmp-dir\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.596236 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.596218    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-model-cache\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.596294 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.596252    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-home\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.597862 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.597846    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-dshm\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.598159 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.598143    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b7337ed-35cd-42ff-85be-e6516693a4e6-tls-certs\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.604184 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.604165    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krskk\" (UniqueName: \"kubernetes.io/projected/0b7337ed-35cd-42ff-85be-e6516693a4e6-kube-api-access-krskk\") pod \"sub-test-model-59ce67-kserve-54f65ccbf-5qbtn\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") " pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.709916 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.709885    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:05.874504 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:05.874480    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"]
Jul 08 23:40:05.876160 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:40:05.876134    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b7337ed_35cd_42ff_85be_e6516693a4e6.slice/crio-779ab872eeb0147d96986fe0c7e94116270c0ddc989431fe665789ccf4c8fa6e WatchSource:0}: Error finding container 779ab872eeb0147d96986fe0c7e94116270c0ddc989431fe665789ccf4c8fa6e: Status 404 returned error can't find the container with id 779ab872eeb0147d96986fe0c7e94116270c0ddc989431fe665789ccf4c8fa6e
Jul 08 23:40:06.167591 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:06.167494    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn" event={"ID":"0b7337ed-35cd-42ff-85be-e6516693a4e6","Type":"ContainerStarted","Data":"330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69"}
Jul 08 23:40:06.167591 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:06.167542    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn" event={"ID":"0b7337ed-35cd-42ff-85be-e6516693a4e6","Type":"ContainerStarted","Data":"779ab872eeb0147d96986fe0c7e94116270c0ddc989431fe665789ccf4c8fa6e"}
Jul 08 23:40:06.184191 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:06.184143    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn" podStartSLOduration=1.18412885 podStartE2EDuration="1.18412885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:40:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:40:06.182419066 +0000 UTC m=+2225.112453042" watchObservedRunningTime="2026-07-08 23:40:06.18412885 +0000 UTC m=+2225.114162824"
Jul 08 23:40:15.710542 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:15.710507    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:15.710918 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:15.710554    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:15.722987 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:15.722961    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:15.768301 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:15.768268    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:40:16.228056 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:16.228027    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:40:50.165995 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:50.165956    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:40:51.081334 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:51.081303    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"]
Jul 08 23:40:51.098035 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:40:51.098002    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:41:01.884615 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:01.884589    2560 scope.go:117] "RemoveContainer" containerID="3ae0674b6928327597803fef1e25279eddc559667bf078ae8006427dd0b62c28"
Jul 08 23:41:06.101590 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.101532    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn" podUID="0b7337ed-35cd-42ff-85be-e6516693a4e6" containerName="main" containerID="cri-o://330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69" gracePeriod=45
Jul 08 23:41:06.217533 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.217504    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn" podUID="0b7337ed-35cd-42ff-85be-e6516693a4e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": dial tcp 10.134.0.58:8000: connect: connection refused"
Jul 08 23:41:06.362053 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.361996    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:41:06.460967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.460936    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-tmp-dir\") pod \"0b7337ed-35cd-42ff-85be-e6516693a4e6\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") "
Jul 08 23:41:06.460967 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.460972    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-home\") pod \"0b7337ed-35cd-42ff-85be-e6516693a4e6\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") "
Jul 08 23:41:06.461219 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.461049    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-model-cache\") pod \"0b7337ed-35cd-42ff-85be-e6516693a4e6\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") "
Jul 08 23:41:06.461219 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.461075    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-dshm\") pod \"0b7337ed-35cd-42ff-85be-e6516693a4e6\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") "
Jul 08 23:41:06.461219 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.461125    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krskk\" (UniqueName: \"kubernetes.io/projected/0b7337ed-35cd-42ff-85be-e6516693a4e6-kube-api-access-krskk\") pod \"0b7337ed-35cd-42ff-85be-e6516693a4e6\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") "
Jul 08 23:41:06.461219 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.461159    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b7337ed-35cd-42ff-85be-e6516693a4e6-tls-certs\") pod \"0b7337ed-35cd-42ff-85be-e6516693a4e6\" (UID: \"0b7337ed-35cd-42ff-85be-e6516693a4e6\") "
Jul 08 23:41:06.461405 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.461262    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0b7337ed-35cd-42ff-85be-e6516693a4e6" (UID: "0b7337ed-35cd-42ff-85be-e6516693a4e6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:41:06.461405 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.461284    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-home" (OuterVolumeSpecName: "home") pod "0b7337ed-35cd-42ff-85be-e6516693a4e6" (UID: "0b7337ed-35cd-42ff-85be-e6516693a4e6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:41:06.461405 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.461363    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-model-cache" (OuterVolumeSpecName: "model-cache") pod "0b7337ed-35cd-42ff-85be-e6516693a4e6" (UID: "0b7337ed-35cd-42ff-85be-e6516693a4e6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:41:06.461571 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.461531    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-model-cache\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:41:06.461571 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.461550    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-tmp-dir\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:41:06.461571 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.461563    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-home\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:41:06.463195 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.463166    2560 generic.go:356] "Generic (PLEG): container finished" podID="0b7337ed-35cd-42ff-85be-e6516693a4e6" containerID="330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69" exitCode=0
Jul 08 23:41:06.463332 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.463235    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"
Jul 08 23:41:06.463332 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.463256    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn" event={"ID":"0b7337ed-35cd-42ff-85be-e6516693a4e6","Type":"ContainerDied","Data":"330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69"}
Jul 08 23:41:06.463332 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.463317    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn" event={"ID":"0b7337ed-35cd-42ff-85be-e6516693a4e6","Type":"ContainerDied","Data":"779ab872eeb0147d96986fe0c7e94116270c0ddc989431fe665789ccf4c8fa6e"}
Jul 08 23:41:06.463535 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.463340    2560 scope.go:117] "RemoveContainer" containerID="330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69"
Jul 08 23:41:06.463535 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.463436    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-dshm" (OuterVolumeSpecName: "dshm") pod "0b7337ed-35cd-42ff-85be-e6516693a4e6" (UID: "0b7337ed-35cd-42ff-85be-e6516693a4e6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 23:41:06.463535 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.463489    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b7337ed-35cd-42ff-85be-e6516693a4e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0b7337ed-35cd-42ff-85be-e6516693a4e6" (UID: "0b7337ed-35cd-42ff-85be-e6516693a4e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:41:06.463852 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.463825    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7337ed-35cd-42ff-85be-e6516693a4e6-kube-api-access-krskk" (OuterVolumeSpecName: "kube-api-access-krskk") pod "0b7337ed-35cd-42ff-85be-e6516693a4e6" (UID: "0b7337ed-35cd-42ff-85be-e6516693a4e6"). InnerVolumeSpecName "kube-api-access-krskk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:41:06.481977 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.481954    2560 scope.go:117] "RemoveContainer" containerID="330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69"
Jul 08 23:41:06.482262 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:41:06.482242    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69\": container with ID starting with 330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69 not found: ID does not exist" containerID="330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69"
Jul 08 23:41:06.482326 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.482276    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69"} err="failed to get container status \"330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69\": rpc error: code = NotFound desc = could not find container \"330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69\": container with ID starting with 330897cee24f0cfa4068c560d0491a7c93e59921a0e3efe70632cff20e113e69 not found: ID does not exist"
Jul 08 23:41:06.562927 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.562892    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b7337ed-35cd-42ff-85be-e6516693a4e6-dshm\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:41:06.562927 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.562920    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krskk\" (UniqueName: \"kubernetes.io/projected/0b7337ed-35cd-42ff-85be-e6516693a4e6-kube-api-access-krskk\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:41:06.562927 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.562930    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b7337ed-35cd-42ff-85be-e6516693a4e6-tls-certs\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:41:06.784252 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.784224    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"]
Jul 08 23:41:06.787821 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:06.787793    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-59ce67/sub-test-model-59ce67-kserve-54f65ccbf-5qbtn"]
Jul 08 23:41:07.705420 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:07.705385    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b7337ed-35cd-42ff-85be-e6516693a4e6" path="/var/lib/kubelet/pods/0b7337ed-35cd-42ff-85be-e6516693a4e6/volumes"
Jul 08 23:41:07.928376 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:07.928346    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:41:39.344608 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:39.344575    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:41:49.806253 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:49.806209    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:41:49.911563 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:49.911516    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:41:51.314830 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:51.314797    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:41:51.415981 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:51.415943    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:41:57.210951 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:57.210857    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:41:57.311456 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:41:57.311419    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:42:30.363174 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:30.363134    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:42:31.230251 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:31.230216    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:42:32.129330 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.129292    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539377230896"]
Jul 08 23:42:32.129740 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.129700    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b7337ed-35cd-42ff-85be-e6516693a4e6" containerName="main"
Jul 08 23:42:32.129740 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.129712    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b7337ed-35cd-42ff-85be-e6516693a4e6" containerName="main"
Jul 08 23:42:32.129812 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.129785    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b7337ed-35cd-42ff-85be-e6516693a4e6" containerName="main"
Jul 08 23:42:32.133081 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.133064    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539377230896"
Jul 08 23:42:32.137997 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.137972    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539377230896"]
Jul 08 23:42:32.256402 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.256369    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqk9g\" (UniqueName: \"kubernetes.io/projected/66d989b4-49e7-4316-b5a8-eb2f29734df3-kube-api-access-fqk9g\") pod \"test-curl-7671-140539377230896\" (UID: \"66d989b4-49e7-4316-b5a8-eb2f29734df3\") " pod="odh-ai-gateway-infra/test-curl-7671-140539377230896"
Jul 08 23:42:32.357712 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.357674    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqk9g\" (UniqueName: \"kubernetes.io/projected/66d989b4-49e7-4316-b5a8-eb2f29734df3-kube-api-access-fqk9g\") pod \"test-curl-7671-140539377230896\" (UID: \"66d989b4-49e7-4316-b5a8-eb2f29734df3\") " pod="odh-ai-gateway-infra/test-curl-7671-140539377230896"
Jul 08 23:42:32.366186 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.366160    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqk9g\" (UniqueName: \"kubernetes.io/projected/66d989b4-49e7-4316-b5a8-eb2f29734df3-kube-api-access-fqk9g\") pod \"test-curl-7671-140539377230896\" (UID: \"66d989b4-49e7-4316-b5a8-eb2f29734df3\") " pod="odh-ai-gateway-infra/test-curl-7671-140539377230896"
Jul 08 23:42:32.443249 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.443219    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539377230896"
Jul 08 23:42:32.572954 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.572930    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539377230896"]
Jul 08 23:42:32.574980 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:42:32.574953    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66d989b4_49e7_4316_b5a8_eb2f29734df3.slice/crio-1db5f3e399f8d9f5896973d476bec7b57a3d4566e4c418fbc6a3be8be0e6cdd5 WatchSource:0}: Error finding container 1db5f3e399f8d9f5896973d476bec7b57a3d4566e4c418fbc6a3be8be0e6cdd5: Status 404 returned error can't find the container with id 1db5f3e399f8d9f5896973d476bec7b57a3d4566e4c418fbc6a3be8be0e6cdd5
Jul 08 23:42:32.576263 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.576248    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 23:42:32.870353 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:32.870265    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/test-curl-7671-140539377230896" event={"ID":"66d989b4-49e7-4316-b5a8-eb2f29734df3","Type":"ContainerStarted","Data":"1db5f3e399f8d9f5896973d476bec7b57a3d4566e4c418fbc6a3be8be0e6cdd5"}
Jul 08 23:42:33.876911 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:33.876873    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/test-curl-7671-140539377230896" event={"ID":"66d989b4-49e7-4316-b5a8-eb2f29734df3","Type":"ContainerStarted","Data":"2dcf7121260170e101da9cfa33cd5102802827139a48e223d1593d2e1f791ad4"}
Jul 08 23:42:33.893526 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:33.893459    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/test-curl-7671-140539377230896" podStartSLOduration=1.17865913 podStartE2EDuration="1.893443476s" totalImagesPullingTime="714.784346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 23:42:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:42:33.891344271 +0000 UTC m=+2372.821378247" watchObservedRunningTime="2026-07-08 23:42:33.893443476 +0000 UTC m=+2372.823477449"
Jul 08 23:42:43.925902 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:43.925863    2560 generic.go:356] "Generic (PLEG): container finished" podID="66d989b4-49e7-4316-b5a8-eb2f29734df3" containerID="2dcf7121260170e101da9cfa33cd5102802827139a48e223d1593d2e1f791ad4" exitCode=28
Jul 08 23:42:43.926286 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:43.925937    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/test-curl-7671-140539377230896" event={"ID":"66d989b4-49e7-4316-b5a8-eb2f29734df3","Type":"ContainerDied","Data":"2dcf7121260170e101da9cfa33cd5102802827139a48e223d1593d2e1f791ad4"}
Jul 08 23:42:45.070784 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.070762    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539377230896"
Jul 08 23:42:45.103965 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.103932    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539377230896"]
Jul 08 23:42:45.107668 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.107642    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539377230896"]
Jul 08 23:42:45.182982 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.182911    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqk9g\" (UniqueName: \"kubernetes.io/projected/66d989b4-49e7-4316-b5a8-eb2f29734df3-kube-api-access-fqk9g\") pod \"66d989b4-49e7-4316-b5a8-eb2f29734df3\" (UID: \"66d989b4-49e7-4316-b5a8-eb2f29734df3\") "
Jul 08 23:42:45.184921 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.184892    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66d989b4-49e7-4316-b5a8-eb2f29734df3-kube-api-access-fqk9g" (OuterVolumeSpecName: "kube-api-access-fqk9g") pod "66d989b4-49e7-4316-b5a8-eb2f29734df3" (UID: "66d989b4-49e7-4316-b5a8-eb2f29734df3"). InnerVolumeSpecName "kube-api-access-fqk9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:42:45.265580 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.265546    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539377232816"]
Jul 08 23:42:45.265972 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.265956    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66d989b4-49e7-4316-b5a8-eb2f29734df3" containerName="test-curl-7671-140539377230896"
Jul 08 23:42:45.266063 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.265974    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="66d989b4-49e7-4316-b5a8-eb2f29734df3" containerName="test-curl-7671-140539377230896"
Jul 08 23:42:45.266063 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.266034    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="66d989b4-49e7-4316-b5a8-eb2f29734df3" containerName="test-curl-7671-140539377230896"
Jul 08 23:42:45.269049 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.269032    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539377232816"
Jul 08 23:42:45.273276 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.273251    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539377232816"]
Jul 08 23:42:45.283940 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.283912    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqk9g\" (UniqueName: \"kubernetes.io/projected/66d989b4-49e7-4316-b5a8-eb2f29734df3-kube-api-access-fqk9g\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:42:45.384796 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.384763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg4zn\" (UniqueName: \"kubernetes.io/projected/7333c705-8aa3-4e5a-b506-075d9bbd94e6-kube-api-access-gg4zn\") pod \"test-curl-7671-140539377232816\" (UID: \"7333c705-8aa3-4e5a-b506-075d9bbd94e6\") " pod="odh-ai-gateway-infra/test-curl-7671-140539377232816"
Jul 08 23:42:45.486236 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.486147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg4zn\" (UniqueName: \"kubernetes.io/projected/7333c705-8aa3-4e5a-b506-075d9bbd94e6-kube-api-access-gg4zn\") pod \"test-curl-7671-140539377232816\" (UID: \"7333c705-8aa3-4e5a-b506-075d9bbd94e6\") " pod="odh-ai-gateway-infra/test-curl-7671-140539377232816"
Jul 08 23:42:45.494662 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.494633    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg4zn\" (UniqueName: \"kubernetes.io/projected/7333c705-8aa3-4e5a-b506-075d9bbd94e6-kube-api-access-gg4zn\") pod \"test-curl-7671-140539377232816\" (UID: \"7333c705-8aa3-4e5a-b506-075d9bbd94e6\") " pod="odh-ai-gateway-infra/test-curl-7671-140539377232816"
Jul 08 23:42:45.579502 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.579450    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539377232816"
Jul 08 23:42:45.704663 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.704633    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66d989b4-49e7-4316-b5a8-eb2f29734df3" path="/var/lib/kubelet/pods/66d989b4-49e7-4316-b5a8-eb2f29734df3/volumes"
Jul 08 23:42:45.712010 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.711975    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539377232816"]
Jul 08 23:42:45.714623 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:42:45.714597    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7333c705_8aa3_4e5a_b506_075d9bbd94e6.slice/crio-f65324ccaf8050649a53a5afe47cfdef8c4c627fbe84fcb3c43616e82bedbfd8 WatchSource:0}: Error finding container f65324ccaf8050649a53a5afe47cfdef8c4c627fbe84fcb3c43616e82bedbfd8: Status 404 returned error can't find the container with id f65324ccaf8050649a53a5afe47cfdef8c4c627fbe84fcb3c43616e82bedbfd8
Jul 08 23:42:45.936954 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.936919    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/test-curl-7671-140539377232816" event={"ID":"7333c705-8aa3-4e5a-b506-075d9bbd94e6","Type":"ContainerStarted","Data":"f65324ccaf8050649a53a5afe47cfdef8c4c627fbe84fcb3c43616e82bedbfd8"}
Jul 08 23:42:45.938075 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.938052    2560 scope.go:117] "RemoveContainer" containerID="2dcf7121260170e101da9cfa33cd5102802827139a48e223d1593d2e1f791ad4"
Jul 08 23:42:45.938075 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:45.938069    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539377230896"
Jul 08 23:42:46.945667 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:46.945630    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/test-curl-7671-140539377232816" event={"ID":"7333c705-8aa3-4e5a-b506-075d9bbd94e6","Type":"ContainerStarted","Data":"32491343ffe7d1468165ae98d04556e4f7abb095945c75da2d96558553e96445"}
Jul 08 23:42:57.000461 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:57.000424    2560 generic.go:356] "Generic (PLEG): container finished" podID="7333c705-8aa3-4e5a-b506-075d9bbd94e6" containerID="32491343ffe7d1468165ae98d04556e4f7abb095945c75da2d96558553e96445" exitCode=28
Jul 08 23:42:57.000869 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:57.000486    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/test-curl-7671-140539377232816" event={"ID":"7333c705-8aa3-4e5a-b506-075d9bbd94e6","Type":"ContainerDied","Data":"32491343ffe7d1468165ae98d04556e4f7abb095945c75da2d96558553e96445"}
Jul 08 23:42:58.146149 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:58.146119    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539377232816"
Jul 08 23:42:58.206531 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:58.206497    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539377232816"]
Jul 08 23:42:58.209881 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:58.209860    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg4zn\" (UniqueName: \"kubernetes.io/projected/7333c705-8aa3-4e5a-b506-075d9bbd94e6-kube-api-access-gg4zn\") pod \"7333c705-8aa3-4e5a-b506-075d9bbd94e6\" (UID: \"7333c705-8aa3-4e5a-b506-075d9bbd94e6\") "
Jul 08 23:42:58.212410 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:58.212380    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7333c705-8aa3-4e5a-b506-075d9bbd94e6-kube-api-access-gg4zn" (OuterVolumeSpecName: "kube-api-access-gg4zn") pod "7333c705-8aa3-4e5a-b506-075d9bbd94e6" (UID: "7333c705-8aa3-4e5a-b506-075d9bbd94e6"). InnerVolumeSpecName "kube-api-access-gg4zn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:42:58.213820 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:58.213799    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/test-curl-7671-140539377232816"]
Jul 08 23:42:58.311118 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:58.311032    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg4zn\" (UniqueName: \"kubernetes.io/projected/7333c705-8aa3-4e5a-b506-075d9bbd94e6-kube-api-access-gg4zn\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:42:59.011784 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:59.011756    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f65324ccaf8050649a53a5afe47cfdef8c4c627fbe84fcb3c43616e82bedbfd8"
Jul 08 23:42:59.011784 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:59.011759    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/test-curl-7671-140539377232816"
Jul 08 23:42:59.021831 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:42:59.021800    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7671-140539377232816\" is forbidden: User \"system:node:ip-10-0-131-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-131-54.ec2.internal' and this object" podUID="7333c705-8aa3-4e5a-b506-075d9bbd94e6" pod="odh-ai-gateway-infra/test-curl-7671-140539377232816"
Jul 08 23:42:59.704957 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:42:59.704916    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7333c705-8aa3-4e5a-b506-075d9bbd94e6" path="/var/lib/kubelet/pods/7333c705-8aa3-4e5a-b506-075d9bbd94e6/volumes"
Jul 08 23:43:01.798412 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:01.798382    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:43:01.802825 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:01.802803    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fcsjl_221668dd-86f4-4e20-bc7f-12cd2d4ce3d6/ovn-acl-logging/0.log"
Jul 08 23:43:11.705223 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:11.705187    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:12.088610 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.088520    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:12.186992 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.186955    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:12.288100 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.288066    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:12.388616 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.388539    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"]
Jul 08 23:43:12.389434 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.389389    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w" podUID="1fce968c-736e-40f3-a7af-d78711729045" containerName="maas-api" containerID="cri-o://fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612" gracePeriod=30
Jul 08 23:43:12.412548 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.412516    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:12.643272 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.643211    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:43:12.751406 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.751378    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx7rf\" (UniqueName: \"kubernetes.io/projected/1fce968c-736e-40f3-a7af-d78711729045-kube-api-access-lx7rf\") pod \"1fce968c-736e-40f3-a7af-d78711729045\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") "
Jul 08 23:43:12.751907 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.751492    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fce968c-736e-40f3-a7af-d78711729045-openshift-service-ca\") pod \"1fce968c-736e-40f3-a7af-d78711729045\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") "
Jul 08 23:43:12.751907 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.751532    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fce968c-736e-40f3-a7af-d78711729045-maas-api-tls\") pod \"1fce968c-736e-40f3-a7af-d78711729045\" (UID: \"1fce968c-736e-40f3-a7af-d78711729045\") "
Jul 08 23:43:12.751907 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.751834    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fce968c-736e-40f3-a7af-d78711729045-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1fce968c-736e-40f3-a7af-d78711729045" (UID: "1fce968c-736e-40f3-a7af-d78711729045"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:43:12.753606 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.753583    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fce968c-736e-40f3-a7af-d78711729045-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1fce968c-736e-40f3-a7af-d78711729045" (UID: "1fce968c-736e-40f3-a7af-d78711729045"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:43:12.753709 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.753677    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fce968c-736e-40f3-a7af-d78711729045-kube-api-access-lx7rf" (OuterVolumeSpecName: "kube-api-access-lx7rf") pod "1fce968c-736e-40f3-a7af-d78711729045" (UID: "1fce968c-736e-40f3-a7af-d78711729045"). InnerVolumeSpecName "kube-api-access-lx7rf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:43:12.852858 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.852823    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx7rf\" (UniqueName: \"kubernetes.io/projected/1fce968c-736e-40f3-a7af-d78711729045-kube-api-access-lx7rf\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:43:12.852858 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.852848    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fce968c-736e-40f3-a7af-d78711729045-openshift-service-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:43:12.852858 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:12.852859    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fce968c-736e-40f3-a7af-d78711729045-maas-api-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:43:13.081179 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:13.081146    2560 generic.go:356] "Generic (PLEG): container finished" podID="1fce968c-736e-40f3-a7af-d78711729045" containerID="fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612" exitCode=0
Jul 08 23:43:13.081361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:13.081191    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"
Jul 08 23:43:13.081361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:13.081193    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w" event={"ID":"1fce968c-736e-40f3-a7af-d78711729045","Type":"ContainerDied","Data":"fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612"}
Jul 08 23:43:13.081361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:13.081248    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w" event={"ID":"1fce968c-736e-40f3-a7af-d78711729045","Type":"ContainerDied","Data":"dbce8e205c1f690e5f192587a6d4bec9cd77298493ad97c9f4a014f677f1e18c"}
Jul 08 23:43:13.081361 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:13.081270    2560 scope.go:117] "RemoveContainer" containerID="fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612"
Jul 08 23:43:13.090809 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:13.090792    2560 scope.go:117] "RemoveContainer" containerID="fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612"
Jul 08 23:43:13.091089 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:43:13.091067    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612\": container with ID starting with fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612 not found: ID does not exist" containerID="fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612"
Jul 08 23:43:13.091137 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:13.091099    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612"} err="failed to get container status \"fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612\": rpc error: code = NotFound desc = could not find container \"fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612\": container with ID starting with fbeaa45d77dc00f817b6b240d7851d9f6c593b29c30b9fab57f83d873de84612 not found: ID does not exist"
Jul 08 23:43:13.102968 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:13.102942    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"]
Jul 08 23:43:13.107437 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:13.107416    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-45d83d-996bd8774-89r2w"]
Jul 08 23:43:13.705438 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:13.705403    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fce968c-736e-40f3-a7af-d78711729045" path="/var/lib/kubelet/pods/1fce968c-736e-40f3-a7af-d78711729045/volumes"
Jul 08 23:43:18.167917 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:18.167888    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:18.232764 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:18.232737    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:18.494601 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:18.494570    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:18.815689 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:18.815594    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:19.217944 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.217896    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:19.318993 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.318956    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:19.421134 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.421092    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:19.511802 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.511723    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"]
Jul 08 23:43:19.511966 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.511940    2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl" podUID="dcfae46a-8f9a-4445-94a8-70da6eaf5eb8" containerName="maas-api" containerID="cri-o://f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907" gracePeriod=30
Jul 08 23:43:19.539529 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.539502    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:19.758957 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.758936    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:43:19.810680 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.810597    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-openshift-service-ca\") pod \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") "
Jul 08 23:43:19.810828 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.810706    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-maas-api-tls\") pod \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") "
Jul 08 23:43:19.810828 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.810771    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4vqn\" (UniqueName: \"kubernetes.io/projected/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-kube-api-access-k4vqn\") pod \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\" (UID: \"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8\") "
Jul 08 23:43:19.810948 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.810925    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dcfae46a-8f9a-4445-94a8-70da6eaf5eb8" (UID: "dcfae46a-8f9a-4445-94a8-70da6eaf5eb8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 23:43:19.811277 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.811254    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-openshift-service-ca\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:43:19.812750 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.812726    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dcfae46a-8f9a-4445-94a8-70da6eaf5eb8" (UID: "dcfae46a-8f9a-4445-94a8-70da6eaf5eb8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 23:43:19.812852 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.812822    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-kube-api-access-k4vqn" (OuterVolumeSpecName: "kube-api-access-k4vqn") pod "dcfae46a-8f9a-4445-94a8-70da6eaf5eb8" (UID: "dcfae46a-8f9a-4445-94a8-70da6eaf5eb8"). InnerVolumeSpecName "kube-api-access-k4vqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 23:43:19.912595 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.912552    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-maas-api-tls\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:43:19.912595 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:19.912590    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4vqn\" (UniqueName: \"kubernetes.io/projected/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8-kube-api-access-k4vqn\") on node \"ip-10-0-131-54.ec2.internal\" DevicePath \"\""
Jul 08 23:43:20.115259 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:20.115171    2560 generic.go:356] "Generic (PLEG): container finished" podID="dcfae46a-8f9a-4445-94a8-70da6eaf5eb8" containerID="f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907" exitCode=0
Jul 08 23:43:20.115259 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:20.115220    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"
Jul 08 23:43:20.115452 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:20.115260    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl" event={"ID":"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8","Type":"ContainerDied","Data":"f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907"}
Jul 08 23:43:20.115452 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:20.115312    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl" event={"ID":"dcfae46a-8f9a-4445-94a8-70da6eaf5eb8","Type":"ContainerDied","Data":"e84a98c2f2d73073226ad24ad96e33f872c656e204fc343d1d574f8fee1430ce"}
Jul 08 23:43:20.115452 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:20.115334    2560 scope.go:117] "RemoveContainer" containerID="f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907"
Jul 08 23:43:20.124353 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:20.124338    2560 scope.go:117] "RemoveContainer" containerID="f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907"
Jul 08 23:43:20.124629 ip-10-0-131-54 kubenswrapper[2560]: E0708 23:43:20.124610    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907\": container with ID starting with f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907 not found: ID does not exist" containerID="f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907"
Jul 08 23:43:20.124683 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:20.124637    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907"} err="failed to get container status \"f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907\": rpc error: code = NotFound desc = could not find container \"f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907\": container with ID starting with f21c353311d37ac27c778ec02b8d4a5b35fa4906a5d91820341646bb6ba94907 not found: ID does not exist"
Jul 08 23:43:20.136341 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:20.136321    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"]
Jul 08 23:43:20.141907 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:20.141888    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-59ce67-76f8ddb796-5trjl"]
Jul 08 23:43:21.704657 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:21.704625    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcfae46a-8f9a-4445-94a8-70da6eaf5eb8" path="/var/lib/kubelet/pods/dcfae46a-8f9a-4445-94a8-70da6eaf5eb8/volumes"
Jul 08 23:43:25.383754 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:25.383692    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:25.454880 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:25.454843    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:25.652691 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:25.652609    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:26.286879 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:26.286845    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f6f84675b-9jg9j_bfee96e5-6fc8-41fb-bef6-30a3b3fd876d/authorino/0.log"
Jul 08 23:43:31.253185 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:31.253130    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cdd595cbc-46p46_0d846b17-e0e1-4a00-937b-f51fd5fdcbba/manager/0.log"
Jul 08 23:43:31.478915 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:31.478877    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-l97bd_2980ea4f-2586-4c63-b48a-ecdd785b07cb/manager/0.log"
Jul 08 23:43:31.593157 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:31.593083    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ccc7894f9-p2gp6_ff97da5f-6b49-418e-b869-8505b2378fe9/copy-manifests/0.log"
Jul 08 23:43:31.601122 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:31.601083    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ccc7894f9-p2gp6_ff97da5f-6b49-418e-b869-8505b2378fe9/manager/0.log"
Jul 08 23:43:32.188761 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:32.188713    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-785d848678-c4frh_f779c33a-fe24-405b-8285-da64afe92edf/maas-api/0.log"
Jul 08 23:43:32.300335 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:32.300311    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-psmps_3b2851d0-57c0-42c1-ac08-56d0148894ba/postgres/0.log"
Jul 08 23:43:33.567308 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:33.567274    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f6f84675b-9jg9j_bfee96e5-6fc8-41fb-bef6-30a3b3fd876d/authorino/0.log"
Jul 08 23:43:33.784375 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:33.784345    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-249lv_67b264e3-af9e-4284-9ba7-a94656ba2ad9/manager/0.log"
Jul 08 23:43:33.893234 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:33.893154    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-p5z5l_d6a7e345-1bc1-44fa-9bf0-a91d25622140/kuadrant-console-plugin/0.log"
Jul 08 23:43:34.918212 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:34.918184    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7dc6c875b-g2ck2_313ba2f0-c401-44e6-a1b4-7e191541c081/kube-auth-proxy/0.log"
Jul 08 23:43:35.398488 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:35.398437    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ksm54_2a55373e-1a33-4adb-a322-54700ce85504/payload-processing/0.log"
Jul 08 23:43:35.854008 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:35.853980    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-89dnw_f0886454-f00f-4d4a-9d7b-87ef2ea73ecd/main/0.log"
Jul 08 23:43:36.310028 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:36.309985    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-67hc9_34bf407c-e983-4019-bf42-fa9b342fd492/main/0.log"
Jul 08 23:43:36.420266 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:36.420221    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-5r99k_a7d29c68-9624-40aa-84a5-9b68184aa2b0/main/0.log"
Jul 08 23:43:40.301142 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:40.301105    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:40.399164 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:40.399124    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:43.171723 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:43.171694    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w7pd9_aa634889-de73-473e-915b-56355ad11639/global-pull-secret-syncer/0.log"
Jul 08 23:43:43.225874 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:43.225828    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log"
Jul 08 23:43:43.316934 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:43.316903    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-54.ec2.internal_35a06a8130ed1bb89742a1b518abf791/haproxy/0.log"
Jul 08 23:43:45.716338 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:45.716292    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r2qwv_14eb9b44-6492-4611-8769-e80480cab2d3/konnectivity-agent/0.log" limit=1024
Jul 08 23:43:48.442874 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:48.442840    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f6f84675b-9jg9j_bfee96e5-6fc8-41fb-bef6-30a3b3fd876d/authorino/0.log"
Jul 08 23:43:48.500778 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:48.500744    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-249lv_67b264e3-af9e-4284-9ba7-a94656ba2ad9/manager/0.log"
Jul 08 23:43:48.528146 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:48.528119    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-p5z5l_d6a7e345-1bc1-44fa-9bf0-a91d25622140/kuadrant-console-plugin/0.log"
Jul 08 23:43:50.636062 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:50.636035    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_172a1281-af1b-4375-9f13-601ea1421bcc/alertmanager/0.log"
Jul 08 23:43:50.659643 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:50.659559    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_172a1281-af1b-4375-9f13-601ea1421bcc/config-reloader/0.log"
Jul 08 23:43:50.682762 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:50.682733    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_172a1281-af1b-4375-9f13-601ea1421bcc/kube-rbac-proxy-web/0.log"
Jul 08 23:43:50.705488 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:50.705444    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_172a1281-af1b-4375-9f13-601ea1421bcc/kube-rbac-proxy/0.log"
Jul 08 23:43:50.730424 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:50.730399    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_172a1281-af1b-4375-9f13-601ea1421bcc/kube-rbac-proxy-metric/0.log"
Jul 08 23:43:50.752860 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:50.752834    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_172a1281-af1b-4375-9f13-601ea1421bcc/prom-label-proxy/0.log"
Jul 08 23:43:50.782345 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:50.782313    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_172a1281-af1b-4375-9f13-601ea1421bcc/init-config-reloader/0.log"
Jul 08 23:43:50.931457 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:50.931430    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-86b868f65f-pbjg2_ee533351-5f8b-461e-b284-af522067952f/metrics-server/0.log"
Jul 08 23:43:51.170114 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.170084    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wdw8h_f05ce760-1a5c-45ca-a213-752db93a15ee/node-exporter/0.log"
Jul 08 23:43:51.190250 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.190168    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wdw8h_f05ce760-1a5c-45ca-a213-752db93a15ee/kube-rbac-proxy/0.log"
Jul 08 23:43:51.213278 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.213256    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wdw8h_f05ce760-1a5c-45ca-a213-752db93a15ee/init-textfile/0.log"
Jul 08 23:43:51.328778 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.328748    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_efdb6df6-0799-4f8a-bcf2-d98df1439561/prometheus/0.log"
Jul 08 23:43:51.352539 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.352511    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_efdb6df6-0799-4f8a-bcf2-d98df1439561/config-reloader/0.log"
Jul 08 23:43:51.380838 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.380809    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_efdb6df6-0799-4f8a-bcf2-d98df1439561/thanos-sidecar/0.log"
Jul 08 23:43:51.404000 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.403975    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_efdb6df6-0799-4f8a-bcf2-d98df1439561/kube-rbac-proxy-web/0.log"
Jul 08 23:43:51.429530 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.429507    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_efdb6df6-0799-4f8a-bcf2-d98df1439561/kube-rbac-proxy/0.log"
Jul 08 23:43:51.454192 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.454117    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_efdb6df6-0799-4f8a-bcf2-d98df1439561/kube-rbac-proxy-thanos/0.log"
Jul 08 23:43:51.479339 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.479312    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_efdb6df6-0799-4f8a-bcf2-d98df1439561/init-config-reloader/0.log"
Jul 08 23:43:51.507379 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.507352    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-lkpl9_7089b439-fd61-4b8a-8cd1-bbf86c74809d/prometheus-operator/0.log"
Jul 08 23:43:51.525662 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.525633    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-lkpl9_7089b439-fd61-4b8a-8cd1-bbf86c74809d/kube-rbac-proxy/0.log"
Jul 08 23:43:51.661354 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.661324    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/thanos-query/0.log"
Jul 08 23:43:51.684175 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.684150    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/kube-rbac-proxy-web/0.log"
Jul 08 23:43:51.710049 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.709972    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/kube-rbac-proxy/0.log"
Jul 08 23:43:51.741870 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.741843    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/prom-label-proxy/0.log"
Jul 08 23:43:51.764209 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.764188    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/kube-rbac-proxy-rules/0.log"
Jul 08 23:43:51.786764 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:51.786744    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c4cd44f8-6krxk_b5a735f9-fd2e-4a5b-8407-0a4f8af80090/kube-rbac-proxy-metrics/0.log"
Jul 08 23:43:53.125569 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.125537    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"]
Jul 08 23:43:53.125946 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.125898    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fce968c-736e-40f3-a7af-d78711729045" containerName="maas-api"
Jul 08 23:43:53.125946 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.125910    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fce968c-736e-40f3-a7af-d78711729045" containerName="maas-api"
Jul 08 23:43:53.125946 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.125925    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcfae46a-8f9a-4445-94a8-70da6eaf5eb8" containerName="maas-api"
Jul 08 23:43:53.125946 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.125931    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcfae46a-8f9a-4445-94a8-70da6eaf5eb8" containerName="maas-api"
Jul 08 23:43:53.125946 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.125946    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7333c705-8aa3-4e5a-b506-075d9bbd94e6" containerName="test-curl-7671-140539377232816"
Jul 08 23:43:53.126119 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.125952    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7333c705-8aa3-4e5a-b506-075d9bbd94e6" containerName="test-curl-7671-140539377232816"
Jul 08 23:43:53.126119 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.126010    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fce968c-736e-40f3-a7af-d78711729045" containerName="maas-api"
Jul 08 23:43:53.126119 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.126020    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7333c705-8aa3-4e5a-b506-075d9bbd94e6" containerName="test-curl-7671-140539377232816"
Jul 08 23:43:53.126119 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.126027    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcfae46a-8f9a-4445-94a8-70da6eaf5eb8" containerName="maas-api"
Jul 08 23:43:53.129089 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.129073    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.131621 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.131597    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rlvz5\"/\"kube-root-ca.crt\""
Jul 08 23:43:53.132433 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.132418    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rlvz5\"/\"openshift-service-ca.crt\""
Jul 08 23:43:53.132517 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.132425    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rlvz5\"/\"default-dockercfg-hrrrl\""
Jul 08 23:43:53.139381 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.139360    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"]
Jul 08 23:43:53.311759 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.311729    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-lib-modules\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.311925 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.311797    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdxmk\" (UniqueName: \"kubernetes.io/projected/fb85c867-6a2d-43f5-bff9-4fc40229327a-kube-api-access-jdxmk\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.311925 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.311829    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-proc\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.311925 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.311887    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-podres\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.312034 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.311927    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-sys\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.413295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.413227    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-lib-modules\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.413295 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.413275    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdxmk\" (UniqueName: \"kubernetes.io/projected/fb85c867-6a2d-43f5-bff9-4fc40229327a-kube-api-access-jdxmk\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.413449 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.413305    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-proc\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.413449 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.413349    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-podres\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.413449 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.413396    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-lib-modules\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.413449 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.413412    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-sys\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.413449 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.413445    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-proc\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.413665 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.413492    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-podres\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.413665 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.413557    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb85c867-6a2d-43f5-bff9-4fc40229327a-sys\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.421753 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.421729    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdxmk\" (UniqueName: \"kubernetes.io/projected/fb85c867-6a2d-43f5-bff9-4fc40229327a-kube-api-access-jdxmk\") pod \"perf-node-gather-daemonset-8fbqq\" (UID: \"fb85c867-6a2d-43f5-bff9-4fc40229327a\") " pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.439822 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.439799    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:53.568592 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:53.568569    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"]
Jul 08 23:43:53.570936 ip-10-0-131-54 kubenswrapper[2560]: W0708 23:43:53.570892    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfb85c867_6a2d_43f5_bff9_4fc40229327a.slice/crio-3ea47f256d6379b8171a9f1ab55d8b24da6c6a6d4a3c6f3025c96ae92f1c320f WatchSource:0}: Error finding container 3ea47f256d6379b8171a9f1ab55d8b24da6c6a6d4a3c6f3025c96ae92f1c320f: Status 404 returned error can't find the container with id 3ea47f256d6379b8171a9f1ab55d8b24da6c6a6d4a3c6f3025c96ae92f1c320f
Jul 08 23:43:54.277134 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:54.277100    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq" event={"ID":"fb85c867-6a2d-43f5-bff9-4fc40229327a","Type":"ContainerStarted","Data":"479b4e23fe417f748dfc3b391f523ce3d8aed6810fc4a62531e15ac1435b0bab"}
Jul 08 23:43:54.279418 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:54.277156    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq" event={"ID":"fb85c867-6a2d-43f5-bff9-4fc40229327a","Type":"ContainerStarted","Data":"3ea47f256d6379b8171a9f1ab55d8b24da6c6a6d4a3c6f3025c96ae92f1c320f"}
Jul 08 23:43:54.279418 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:54.277266    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:43:54.294884 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:54.294832    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq" podStartSLOduration=1.294817356 podStartE2EDuration="1.294817356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 23:43:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 23:43:54.293249374 +0000 UTC m=+2453.223283348" watchObservedRunningTime="2026-07-08 23:43:54.294817356 +0000 UTC m=+2453.224851329"
Jul 08 23:43:55.783356 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:55.783322    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l799w_e1cec76f-e1c9-4ad1-90d7-e0645d42e834/dns/0.log"
Jul 08 23:43:55.806631 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:55.806605    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l799w_e1cec76f-e1c9-4ad1-90d7-e0645d42e834/kube-rbac-proxy/0.log"
Jul 08 23:43:55.909458 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:55.909434    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tkl7k_62bd284c-a66b-43b4-ae40-b1b0022b355b/dns-node-resolver/0.log"
Jul 08 23:43:56.492408 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:56.492385    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g669b_2be52dd7-7972-4006-a2e9-45b4518167ef/node-ca/0.log"
Jul 08 23:43:57.660713 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:57.660686    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7dc6c875b-g2ck2_313ba2f0-c401-44e6-a1b4-7e191541c081/kube-auth-proxy/0.log"
Jul 08 23:43:57.925480 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:57.925444    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ksm54_2a55373e-1a33-4adb-a322-54700ce85504/payload-processing/0.log"
Jul 08 23:43:58.745953 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:58.745918    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-smf48_efb5cd57-0c10-4b3e-9d88-0bdf9d2f22bb/serve-healthcheck-canary/0.log"
Jul 08 23:43:59.271272 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:59.271243    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-56gmw_749c95ca-a7ba-41bf-a894-3fcf44d9de75/kube-rbac-proxy/0.log"
Jul 08 23:43:59.293178 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:59.293154    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-56gmw_749c95ca-a7ba-41bf-a894-3fcf44d9de75/exporter/0.log"
Jul 08 23:43:59.316052 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:43:59.316028    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-56gmw_749c95ca-a7ba-41bf-a894-3fcf44d9de75/extractor/0.log"
Jul 08 23:44:00.291937 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:00.291910    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rlvz5/perf-node-gather-daemonset-8fbqq"
Jul 08 23:44:01.982763 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:01.982734    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cdd595cbc-46p46_0d846b17-e0e1-4a00-937b-f51fd5fdcbba/manager/0.log"
Jul 08 23:44:02.112909 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:02.112880    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-l97bd_2980ea4f-2586-4c63-b48a-ecdd785b07cb/manager/0.log"
Jul 08 23:44:02.140382 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:02.140352    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ccc7894f9-p2gp6_ff97da5f-6b49-418e-b869-8505b2378fe9/manager/0.log"
Jul 08 23:44:02.168326 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:02.168304    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ccc7894f9-p2gp6_ff97da5f-6b49-418e-b869-8505b2378fe9/copy-manifests/0.log"
Jul 08 23:44:03.404189 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:03.404163    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7dc6c875b-g2ck2_313ba2f0-c401-44e6-a1b4-7e191541c081/kube-auth-proxy/0.log"
Jul 08 23:44:03.570066 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:03.570035    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ksm54_2a55373e-1a33-4adb-a322-54700ce85504/payload-processing/0.log"
Jul 08 23:44:03.676116 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:03.676085    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65f9849997-8hwcl_10888b06-ca90-4856-99b0-6c80cea2889d/manager/0.log"
Jul 08 23:44:04.864850 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:04.864778    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7dc6c875b-g2ck2_313ba2f0-c401-44e6-a1b4-7e191541c081/kube-auth-proxy/0.log"
Jul 08 23:44:05.041197 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:05.041160    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ksm54_2a55373e-1a33-4adb-a322-54700ce85504/payload-processing/0.log"
Jul 08 23:44:10.236314 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:10.236287    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw7ph_3681f3f9-bf81-41c8-8caa-75b5123fe91b/kube-multus-additional-cni-plugins/0.log"
Jul 08 23:44:10.278256 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:10.278227    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw7ph_3681f3f9-bf81-41c8-8caa-75b5123fe91b/egress-router-binary-copy/0.log"
Jul 08 23:44:10.347604 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:10.347578    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw7ph_3681f3f9-bf81-41c8-8caa-75b5123fe91b/cni-plugins/0.log"
Jul 08 23:44:10.382250 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:10.382215    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw7ph_3681f3f9-bf81-41c8-8caa-75b5123fe91b/bond-cni-plugin/0.log"
Jul 08 23:44:10.419239 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:10.419204    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw7ph_3681f3f9-bf81-41c8-8caa-75b5123fe91b/routeoverride-cni/0.log"
Jul 08 23:44:10.444664 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:10.444634    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw7ph_3681f3f9-bf81-41c8-8caa-75b5123fe91b/whereabouts-cni-bincopy/0.log"
Jul 08 23:44:10.469513 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:10.469488    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw7ph_3681f3f9-bf81-41c8-8caa-75b5123fe91b/whereabouts-cni/0.log"
Jul 08 23:44:10.688575 ip-10-0-131-54 kubenswrapper[2560]: I0708 23:44:10.688546    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lm4nq_6cb8ea5f-623b-42f4-8345-aedda4457469/kube-multus/0.log"